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ABSTRACT 
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decisibhs bf the Ihterhatibhal Mathematics Committee; (2) tb prbvide 
a historical. recbrd bf the development bf the Secbhd XEA Mathematics 
Study; and (3) tb provide ihf brmatibh ahd guidelihes_f or Natibhal 
C^hters and National Mathemat^ Committee members. The Secbnd lEA 
Mathematics Study, which is an investigation of the teaching and 
learning of mathematics iii schools , consists bf three main 
components: a curriculum analysis, an ekamination. of classroom 
processes, and an analysis of cognitive and -affective student 
outcomes in light cof the nature of the curriculxam and instructional 
practice. Previous bul have described the background of the 

Second lEA Mathematics Study, reported on vari^ 

concerningthe study, and elabpr on the study's design. This 

bulletin describes the updated model for the study ^ gives result of 
pilot testing of the: cognitive instruments us on the 

sampling and^methodological issues associated with the two proposed 
versions (cross-sectional and iongitudi of the study^ and gives 

further_timetable /amendments to allow for a full pilot study of the 
longitudinal version^ (JN) 
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1 . ' Iritrbdtietioh 

I, introduction 

The Mathematics Study Bulletin series of the Iriterriationai Associa- 
tion for the Evaluation of Education Achievemerit (lEA) is intended to 
serve a number of purposes: , 

to consolidate the decisibris of the Ihternatibhal Mathematics 
Committee, 

To provide a historical record of the development of the br 6- 
ject. 

To provide informatibri arid guideliries for National Centers 
arid National Mathematics Committee members, 

.1 Although the buMetins are written prlmirily for use within the 
framework of lEA countrjes and committees, they m^y at times be found 
tlsefuT for gerieral informational purposes, 

' ^ Three presvious Mathematiqs Study bulletins have already been pub- 
lished. Bulletin No^ 1 (Qctober 1976), described the backgrbUrid arid 
evolution of the Second I EA Mathematics Studyi _the_ issues to be ad- 
dressed > arid the procedures proposed to address those issues, and 
. outliried a tentative timetable for the Study, 

- ' Bulletiri No.: 2 [Septeiriber 1977); report^^ con^ 
cerning the Study „ iricludirig ftiridirig and ^organizational^ details, dis- 
cusseid activities of the Iriterriatiorial Mathematjcs Commuted, provided 
an internatidrial. grid summarizing iriitial responses from countries about 
their mathematics curricula^ arid pr:eserited ari updated timetable for the 
Project, V 

Bulletin No, 3 (becembjer 1978}, elaborated upbri the design for 

the Study , provided a detailed timetable, arid summarized sampjina and 
instrument speciflcatidns; The dbcumerit also cbritairied sufficlerit infor- 
mation for Natlorial Centers to prepare prbpbsals Varid proceed with other 
arrangements preparatdry td participatibri iri the Study, 

. This^ builetiri describes the updated model for the Study, gives 

resglts of pilot tesyng of the cdgriitive iristrumerits, reports ori the 
samp' frig arid methodologicai issues associated with the twb proposed 
versioris [cross-section jongitudinal) ,df the study arid gives 

furtheV timetable amendments to allow for a full pilot study of the lori- 
gitudinal. versior^. The resujts of the attitude trials tbgether with the 
recommended Items for the attitude scales for the j:rbss-sectiorial study 
ace included In a separate pubMcation, ^Results bf the Attitude Trials" 
(December 1979). 
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2.. Activities duNrig 1979 
. 2il Methbdblbgy Seminars 

The Second Ihternatidhai: Mathematics Study raises many 'met^^^^ 
logical issues > preddminaht among them being the matter of student 
growth ' in achievement, during the school year. How should growth be 
; measured? What anaiyses shall be employed to account for growth? 
What are the impllcatiqns of thjs interest |n^ growth for^s for 
the structure of the instruments, pjarticularly the cognitive test? 

As a response to this^ important_ probjem^^ methodolo- 
gists from various countries including dap^n, Sweden , Aastralia, New 
Zeajand^ Canada and the States were invited to prepare papers 

to_ address .the growth issue. > 

; Early [n the planning, [t b^carne evident that it was impractieable 
to attempt to gather together the principle participants in one location; 
< Hence,^ it was decided to eohvene two somewhat parallel sessions: one 
at Michigan State University, USA, and another at the AtistraJiarL Gouh- 
cil for Educational Research, under the directibri of John R. Schwille 
and John P. Keeves, respectively. 

\Michlgan State Seminar: February 1-2, 1979 

Overview of the Secbhd lEA Mathematics^ Study. - 

K. J. Travers> University of Illinois 

Overview bf Descriptive Analyses for the Study, 

E. Kifer> Uriiyergity of Kentucky 

Relating Classroom Process Variables td' Measures of Student Achieve- 

merit, arid Leafhihg in the Second I EA Mathematics Study. 

: ' Graham Nuthalj, University of Ganterbury 

Nuthall discussed a number of apparent coritradlctlons^^^ 
idgic of relationships that exist between sfgnif[cant d 
variabies » and the analytical procedures wh ich are available and ad vo- 
cated a procedure based bh student ^B.erfoj^mance on indiyid items ^ 
He arjued that the ap^proprjate form of summary aggregation of data is 
across students, and not across it^ms; using the item scores as records 
of what is known, or not known, rather than as measures of a variable. 
The paper was presented by Professor Kifer in Professor Nathalies 
absence. 

Explanatory potential of teacher topjc questionnaires ' . _ 

: Curtis ^ McKnjgh University of Illinois 
^ Mck^n^ight analysed the specific questionnaires with a view to 

identifying variables possessing explanatory power. 

Test construction issues - A practicar problem and a proposed solution 

'Richard Mt Wolf, Teachers College, 
r"" Columbia University 

Wolf discussed the problem of needlhg a large number of items to 
sample , the proposed . Ihterhatiohal Grid while having only a limited 
testing time available In schools i He examined alterfiative sblutions and 



advocated muitipie matrix sampling as a viabie solution, outlining the 
advantages arid disadvaritages of the method. 

Structural Equations *ancl muitlvarlate least squares; fitting different 
methods to different questions ; 

~ _ W. _ Schmidt^ _M_ichigan State University 

Schmidt contrasted multivariate least squares techniques^ includihg 
, repeated measures models, and structural eqjjations mbdets based dh 
maximum^ likeiihood procedures ._ An expianatipjn of the two methods and 

.examples of research questions pose^ in the study for which the 

methods would be appropriate^ were presented. 

A general, model of task learriirig^^ 

^ Richard ^. Noonah, University of 

Stockholm j ^ , _ 

Nooriari *presesnted a task learning model devised to attempt to solve 
the prpbierar given a data set infcluding ^a variety, of student ^toack- 
; ground, affective and ebgriitive measures, as well as teacher and school 
measures at two points in time [iwe. , beginriirig arid end of school 
year) , how can the data be analysed to shed . light bri the process of 
cognitive growth? 

Mathematics achievemerit iri lorigitudirial models Usirig ridge regressidri 

estimatibh procedures - 

J. vy. Bulcock _arid W.F. Lee, Memdrial 
Univ^rsityi NewfdUridlarid, arid W.S. Luk> 
Simbri Fraser Uriiyersity# Cariada. 
In the paper preserited, the limitatidhs of least squares regressipri 
arid twd stage least squares prdcedures are e:^cfm|ned. Simple ridge 
regressidri is examiried as a pdssible sdlutidri but it is fourid that the 
^ miriimum mean square error is not a proper criterion. However, the 
variance riormalizatiori- cHteridn. is found to correct all the ilmftations of 
simple ridge regression with the former critierion. When simple ridge 
regression using the variance -normalization criterion is used in qonjunc- 
tion with two stage least/ squares, new insights into the reciprocal . 
structure of the cognitive, affective cind; conative outcomes of schooling 
are'ganriedi ^ _ '^'\ 

SeveF'al implications of these findings - both substantive arid 
methodological - for the Siecond lEA Mathematics Study were discussed. ' 

The TEA bohgitudinal analysis frdm a muJtil^ , . 

disentangling ; between-class and withiri-class relatibriships _ _ _ 

Leigh Bursteiri, University of Califorriia, 
Los ^rigejes arid Robert Li Liriri, 
University of Illinois ^ 
The authors discussed how Voe multilevel character of the data 
should influence the lohgitudirial analysis iri the Secdrid lEA Mathematics 
Study and described how a multilevel perspective ih the speeificatiori of 
substaritiye cjUestibris and iri the analysis strategies emplbyed cari pbteri- 
tially clarify the viays in which iristructiorial" practices affect student 
: perfarmarice. They butlined the way iri which a multilevel arialysis of 
the Study data might proceed, arid preserited both the general qUestibris 
that the rriUltilevel arialysis cari address arid specific featura^ df a 
possible arialytical framewbrk. . ^ : 



blscussahts for the sympdsjum were: 

Annagret Harhlsehfeger (CEMRELl ; 

David Berliner (University oX Anzona) 

Andrew Porter (Michigan State University) 

Australia*" ebu hcil for Education Res earch Se minar ;. 
Eebmary 15-20. 1^75 "~" 

The seminar referred to many of the papaers deliver^; at the 
Michigan State meeting and received presentations by the following: 
Rosier, Malcolm. "Planned analyses for the__ Second I EA Mathematics 

Study-Crbss-sectional Study in Australia.^' February 19?9. 1 
"Report of the IE A Second International Mathematics Study Data Analy- 
sis Seminar." Melbourne, AGER, February 1979. t i . 
Keeves J.P. and R. Lewis. "Teachers. , classrooms and studen. out- 



Rosier?"^!?.* "Hypotheses for the Australian National Study of Mathe- 
matics Achievement." " ' 

Capitalizing upon the visits to Australia of Neville Postlethwaite, 
Roy Phillipps, Robert Garden, Ian Livingstone, and Roslyn Slemmt, 
extensive work was done oh the sampling manual and aGministrative 
manuals for the reduced Ceross-sectiohal) study.; An impressive amount 
of work was done on those documents durlna a shorty period of time, 
and drafts were sent for national comment in March-April. 1979. 

The International Mathematics Committee ^benefitted greatly from the 
proceedings of both seminars, and is very appreciative of the progress 
made oh the manuals, as well. 

2.2 lEA General Assembly, Paris, September 17-21, 1979 

This important meeting provided a second op pbrtuhity for the IMC 
to interact directly with representatives from each country plannjng Jo 
take part in the Study (the first opportunity was at the Tokyo Assem- 
bly in January 1978^. Activities during the week-long meeting included: 

2.2.1 Review of the cross-sectional and Ibrigitudinal aspects of the 

Study - _ 

2.2.2 Consideration of planning for the Curriculum Symposium, 

January 1980. - . , ^ 

2.2.3 Consultation with Dr. Keeves bf the International. Sampling 
Committee on a country-by-cbUhtry basis. = 

Iridividuar problems cdhcerning sampling were discussed an^ 
rtlehdations for proceeding were offered. The IMC acknowledges with 
gratitude the tremendous commitment of time and energy offered by Dr. 
Keeves to this important task. 

2 i n Critique of al[ draft final ihstruments. and manuals 

* One impbrtant outcome of this activity was Jhe formation or a 
subcommittee consisting of representatives of the National Mathematics 
Committees from France, Luxembourg, the Netherlands and French and 
...Flemish Belgium, for the. purpose of recommending the inclusion in the 
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cognitive tests for both Populations of additional. Items The items, iri 
the I- category (important ih some countries} would serve to inbre 
accur^ely reflect significant diversity arid emphases in the curriculum^ 
particularly vyfth respect to geometry i As a result, it was agreed by 
the iMG-to increment the rotated forms _ by two items per. form with N 
items- ThUs/ 8 items were added at Population A and 16 items werl 
added at Pbptilatiori B. 

2.2-5 Election of chairman of Math ^ ^ , 

Mr. Roy W. 'Philljpps was elected UhahimdUsly to a second term as 
chairman of the CoUricil. 

2-3 Meetings of , the Ihterhational Mathematics Committee 

2. 3. 1_ Michigan State Meetings -Jan^ary^^T ^bruary 3 ,^a3li 

This meeting took place during the depths of wm 
western United States and featured such ^unpjanned events as the 
"airlifting" of Roy Phillipps arid A. i . Weinzweig from a s^mal^ 
Chicago during a record bjizzard ojne of the Un|yersity of iijinois' 
private airplanes. AM [n aJU the meeting was productive, arid owes Its 
''success largely to the attention paid by Jack ^chwH every detaij. 

The IMC expresses its sincere thanks tcx him and to Michigari State 
University for hostmg this meeting.- Those atteridirig: E. Kifer; S» 
Hilding; R. Garden; J. Wilsori; K. Travers; R. Phillipps. Iriyited 
guejits inciuded F. van der Blij, University of Utrecf;it^ the Nether- 
lands; J. Scfm and occasiorial visits from other scholars iri the 
Michigan State University commuriity. Gerard Pollock was uriable to 
attend due to iliness. 

It was a dlstirict pileasure |b welcome Dr. van der Blij to the 
meeting. His probing questibris arid Hvely cbmmeritary cbritribUted much 
to the substance arid erijbymerit bf the meeting. 

(.1 , , , . • " 

Major accbmplishmerits of the meeting included: 

2.3.1.1 Preparation bf firial forms of the cognitive instruments for the 
reduced (crbss-sectiqrialj study. These were sent out for 

_ : national cornmerit in .March 1979. 

2.3.1.2 Preparatidri of draft background questionnaires for school^ 
teacher arid student for the reduced study. 

2.3.1.3. Review of reQornrTiendfed final forms of the attitude scales for 
the reduced; study. . ; _ 

2.3.1.ij. Review of plans for the scurriculum analysis, including the 
Bielefeid symposium on the curriculum.^^ 

2.3.1.5. bevelbpment of strategies for coordj the^^yar[ous aspects 

of the study (reduced or cross-sectional component, fuM or 
IdrigLitudioal cpm^^ respect to both Populations) . 

These details are elalDorated upon^ m Here, 
it is stated briefly that it appeared rea[i^ tic to proccied ^ith 
Population reduced study, on the schedule as announced 
[n Buiietm 3. Tjiat Js, data colleqtiori could proceed as early 
as* May 1980.^ The ant^ldpajed work to 
give confidence in that projection. However, Populatiori A 
was more problematic. Majbr issues relating tb the cbgriitive 
iristrumerit arid' the classroom prbcess questibririaires vvere yet 



J"eso[ved. FyT*^'09_ instrameht developrnent con- 
tinued to be a severe problem^ The JMe agreed that in view 
of these anticipated djfflcujties, countries shquicl^ where 
possible^ delay data coljection antll 1981, For those few 
' countries which must proceed with data collection for the 
. . reduced study. Population A, in 1980, the eommittee would 
strive to provide Ihterjm instruments arid mariuals by the erid 
: . of 1979. 

2.3.1.^; Eorisjderatlori of jnethddolbglcal issues raised at the Michigan 
State University Symppsiurrii 

\ The .Cbmmittee met for half a: 'day following the symposium to 
review the deliberatibris of the syhijDbsium and putline thg riext stejDS to 
attempt to resolve the many issues raised. Many of the subsequerit 
section^ of this Bulletin reflect the decisions made at this meeting. 

2. 3. 2_ Budapest fAeetirig; September 2^-28^ 1979 ^ 

This meetirig was hosted by the, Ofszagbs Pedagpgiai Iritezet^ 
Wikldsvari Sandor/ Director. Local arrangements were by Zoltan 
Bathoryv the J EA Councy Member for Hungary. The IMC is grateful to 
both gentjemen,, and to Dr. Julia SzendraJ, Dr. Judith kadar-FuIop 
and the many other individuals at the research institute who did so 
much to make the meeting productive and enloyable. Outcomes of this 

meeting included: 

# . • - - ■ • ■ 

2.3.2.1 Finalizing of the instruments (cross-sectfonal study) ^ 
Versipn yi of the draft^ 1979) of the 
Population A cognitive test was adopted for use In the cross- 
sectional study, wth the addltion^ of the suppjementary„ items 
as noted ui7der 2.2.4. For the Population B test, it was 
rioted that for psyehbmatrfe reasoris it is preferable to have 
Items assigned tp forms raridotnly with stratification bri^cbriterit 
arid behavtbr. The versibri bf the Pbpulatlbri B test described 
jri Bulietlri 3 (page 2gj Ls_ribw available as an iriterri^tibrial 
option for the LbngitUdirial Study. ' 

The firial Jbrrri bf the. attitude scales was recbmmerided iri a 
paper by E. Kifer of the IMC and was published separately in 
December 1979, A short scale pri. cbmpUters arid mathematics 
has been added. : 

The_ qui^tiohnaires for school, teacher and student were 
' finalized. Robert Garden of New Zealand an^ David RPbitaille^ 
Cariada/ cbritributed much to this work. Their assistance is 
greatly appreciated. : / 

2.3.2.2 Editing of the manuals : 

Considerabie re-working of the manuals was required in the 
light of developments since the work sessions in Australia in 
February of^ 1979. Much^^^^^^ work at the Budapest meet- 

ing was done by Jack Schwilje of Michigan State University 
and Richard Wolfed dnt^ for Studies in Education. 

NeyiJIe posllethwaite^ Chai^r^^^ of lEA^^ was also able t^ 
attend ; the meeting, for^ several days^ and assist in the task. 
To all of these gentlemen, tjie lMC is greatly indebted for 
their dedicated; and skillful effiDrts. 



2i4 UjDdate: Latin Americari activities 

The Caracas meeting, held in May 1978^ and reported in Bulletin 
3, page 3, dpcUmehted the existence of a broad basis of ihterest in the 
;; Study on the part of mathematics educators in Latin America. A follow- 
Up meeting to consider the next steps for participation in the Study 
was held subseqUeht to the _Fifth IhterAmericah Conference on Mathe^ 
rriatics Educatibh, held in Brazil in February 1979. Limited travel 
monies for this meeting were_ made_ available by the _Organizatjbri of 
American States through the office of Dr. Raul Allard/ Director of. Edu- 
cation. / ' 

Under grant from the Organizatldn of American States, Mr. Peter 
Staples, a graduate research assistant in mathematics edUcatidh at the 
University of Illinois, was efiabled to spend one month in Brazil assist- 
ing with preparatory work for the Study. Mr. Staples also visited the 
Dominican J^epubMc, where Dr. Eduardo Luna is actively involved in 
pilot testing the classroom process instruments. 

ChHe, w]th long experience in lEA^ is well organzed. for participa- 
tion in the^,Study. Other countries^ notabjy_Brazi[^^ Rica and the 
Domihican^^epublid, have^j^^ considerable progress toward participa- 
tion. Presently, mechar^isms are ^ejng^ ejgDlored for promotm^ communi- 
cation within Latin America arid providing ' technjcal consulting services 
as heeded. However, external furiding Is likely to be needed for reallz- 
irig these mecharilsms. 

. " ■ - V. ^ - ■ - ^ 

' - ' ■ , . ft 

. 2.5 Iriternatibrial Fundirig 

*■ , '• ^ 

Mairitenahce of the Office of the Iriternatibrial Cbordlriator Is pro-^ 
yided by the Department of Education; Wellirigtori, Fiew Zealarid; Sup- 
- port for meetirigs bf the Iriterriatibrial Mathematics 'Committee arid for the 
pffide of ^ the Chairmari of the Iriterriatibrial Mathematics Committee cori- 
. tin lies tb__be provided by the Natibrial IristitUtebf Educatiori; U .S. A . 
During 1978-1979^ a large pbrtibri bf the fcpsts bf developing the class- 
room processes instruments for Pppulatibri A were bbrrie by ari. NIE 
. grant- to the U.S. National Mathematics Committee. 

^ The Federal Republic^ of Germany has provided a grant to Dr. 
Hans Steiner, Institute of Mathematical Didactics^ Biele^ for the 
Curriculum Symposium scheduled foV January 7-11, 1980. 

^ .The Ford Foundation prjjvide a small grant to enable the publica- 
tion of a brochure descr[bing the Second Mathematics Study in general 
terms. ^ This grant also allowed the lnternaJ:ional Coo rdmatpr to attend 
a meetirig of th^ JEA Stariding Commlttee_and to visit national centers in 
Hong Kong, Federal Republic of Germariy, Spain, Ireland and the 
United States* 

Two m^jdr needs essential for the completlori of the Stud^ have yet 
tb be met: (1) support for trairiirig of the national research coordi- 
riators, (2J support for interriptibrial data processirig iri New- Zealarid 
arid th^. Uriited States. Dr*' Neville Postlethwalte Is seeking funding for 
these aspects' bf the Study. 



11 



3^ Prbjected activities for 1980 



3.1 \i£drricalum Symposium, January 7-11 / 1980 , - 

thls sympositim, ah essentia! cdmpdnent of the curriculum^analysi^s 
component of the Study^ '^ii! be hosted by the Institute of MathemaU^^^ 
Didactics, Uriiversity of Bielefeld^ Federal Republic of Germanyt Dr. 
Hans Stelner is cbbrdinatbr of the symposium and |n charge of loeaj 
arrangements. PlaJihihg , for thi meeting has been done by the Cur- 
riculum Analysis CrbUp.* Details are given under Section 6, below, 

3.2 Research Cbbrdihatdrs Meetings ' t^t 

3.2.1 Crbss-sectional study, January 1980 

It vvas hoped that a training : sessjon for countries participating m 
the crbss-sectibhal study could be held in January, 1980^^the week 
foMbwing the curriculum symposium. Dr. Steiner kindly agree^Jip make 
the Ibcai arrangements at the Institute of Mathematical Did|^cticSi, 
Bielefeld. However, at the time of writing, it was not known whether 
funding would be available to make this meeting possible. 

3.2.2 Longitudinal Study, December 1980 ' ^ . 

An invitation has been extended tb IEA by the Institute National 
de Recherche Pedagogique, Paris, for such, a session. The jnterna- 
tionai Mathematics ebmmittee is mbst appreciative of this continued 
expression of interest and suppMDrt bh th? part of M . Jaquenod, Direc- 
tor and^M. Daniel Robin. It is expected that the support wjlj provide 
per diem expenses for invited participants. It will "be necessary for 
individual countries to provide their own travel costs to this meeting. 

3;3 lEA General Assembly, Finland, August 4-8, 1986 

This meeting, which provides the oppprtunUy for lEA members to 
deliberate upon general cdhcerns of the Association, also serves as an 
extremely important mechanism for providing Jnteraction between mem- 
bers of the Matiohal Mathematics Committees and the International Mathe- 
matics Cbhrimittee. ; 

^ A J<ey item on the agenda of the iequncij meeting will be a corisid^r- 
atibh bf restJlts of piiot testing of the classroom process iristruments Jh 
these countries: France, Japan, Scotland, Thailand, United States, 



*The Curriculum Analyses iSroup, based at the University of Illinois, 
Urbana, consists of James Hirstein,_Hbracib Pbrta and lari Westbury, of 
the University of inlnois at Urbana-Champaigri, augmented at different 
times by Aian PLirves of the University of Illinois at Uhbana-Champaign; 
A.l. Weinzweig of the University of lljinbis at Chicago Circle; Hans- 
Georg Steiner of the University of Bielef^ld^ FRG; Ulf Lundgren of the. 
Stockholm Institute for Education, Sweden; Ed Jacbbsen, Paris, prance; 
and Roy W. Phillipps^ Department of Education, New Zealand. Kenneth 
Travers serves^ as liaison .between the Curriculum Analysis Group and 
the I riternational Mathematics CbmTO -i ?s 
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together with India arfd the Bojninjcan Repablic throagh indivldaal 
arrarigemehts with the latter two coantrles. 

3.4 Fourth Jnternational Gongress on Mathematics Edacatiori, Berkeley; 
California, USA, August 10-15., 1980 

This impoTtaht/ ioiernatlonal meetih Is held qtiadrehhlally. As- 
jDects of the Second Study will be dealt with bh the program. One 
session is plahhed bh the Curriculum Analysis and ahbther is scheduled 
to report bh results bf pilbt SLtudies bf the classrbbm jDrbcess instru- 
ments in the seven cbuhtries referred tb in 3.3. 

3.5 Other fprthcbmihg iriterhatibhal meetings bf sjDecial iiiterest tb 
mathematics educatdhs. . ' 



The_ First Ihterriatidnal Cdrigress dh the Teaching df Statistics, 
August 1982; the Sixth Inter-American Cdnferenc^^ on Mathematics 
Education, Latin America, 1983; the Fifth International Congress on 
Mathematics Education, 1984, in Australia. 

3.6 Meetings of the International Mathematics Committee i 

It Is expected that the Committee will meet at least twice Jn 1980, 

aUho^agh ftnd^ln^^ not perm[t more than one m^eetjng jh addition to 

those made possible by^ other ^tudy activities, such as the Curriculum 
Symposium In West Germany In January 1980 and the lEA Assembly In 
Finland In August I980i 

A main cbrisjderatibri bf the IMC in 1980 will be the develbpmerital 
wbrk required in preparatibh for the^lbngittidihal study for which data 
collection is scheduled to begin in 1981. 
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4, Purposes arid beriefits of the Study * 

The Second Iriterriatioriaf Mathematics Study is ari irivestjgatibri of 
^ the teaching and learning of mathematies iri schools. The Study has 
three main components: a curriculum arialysis, which is an examiriatidri 
oft the curricular cbntexts in which teaching arid J^earriing take place; a 
study of classroom processes, iritended to provide irifdrmatidri dri what 
takes, pjace iri the classroom as mathematics is taught; and an analysis 
of studerit outcomes (cogriitive arid affective] iri the light of the nature 
of the curriculum arid iristructibrial practice* Twd pdpulatibns are 
targeted for study: 

• PbpUlatibri_ A: All students in the grade enrolling the modal 
riumber bf students which have attained the age of 13.0 - 
13.11 by the middle bf the schddl year. 

Pdpulatidh B: All students who are in the normally' accepted 
termirial grade of the secpndary educational^ 
are studying mathematics as a subs^ant|ajv^par^^^ (approximately 
five hours' per week) of their academic program. 

instruction, even in one subject such as jnathematics, is both 
cdmplex and of crucial importance. Suc^^^^ a highly complex 

network of many components and relationships. The system Is dedi- 
cated to^ the implementation and actualization of the couritry's educa- 
tional intentions In the subject matter area^ If the subject matter is 
impoj^ant to th^ and its cltlzeris, arid heriee if its educatibrial 

intentions are Important to t^he country; theri the iristructibrial delivery 
system becomes crucially Importarit as thfe vehicle for realizirig those 
intentions. . ■ 

Thus the riature bf the system, its structure^ efficiency arid 
power, become presslrig Issues. The cbriceptualizatibri bf the, issues 
Involved will deperid upbri the pbsitidri frdm which dne perceives the 
system^ For the teacher br schbbi oLfficial the issues include the . ade- 
quacy of resources prbvlded, the feasibility, bf tasks assigned, the 
riature bf the erivirbrimerit made pdssibl^^ arid the cdncrete criteria of 
the aehievenierit arid growth bf childreri arid~the-q.uality of educational 
everits. Fbr the researcher the issues are the basis for (arid^^^^^ 
the motivation for} the sdieritific task df authenticating a valid ' model of 
such a system/ expldririg the cause arid effect relationships of the model 
arid~th§ effect biPva Hat^dri-m-4<^y--^ deter- 
miriirig the structural properties and dynamjcs of the^ class of modejs 

exemplified by the giveri Instructional delivery system. For the cur- o 

riculum specialist and for natjon^al (or ijriternati^^ 

makers there are policy questions iuch^s t^he adequacy of outputs bf 
the system In terms of national^ goa[s, the maximizatiori of the efficacy 
of the system^ arid the isolation of parameters that erinarice control arid 
use bf the system. 

One element common to all bf thisse tasks arid issues is ^tKe iristruc- 
. tlonal delivery system that is the focus of each, [rivblved iri each bf 
t^ose tasks Is jhe portrays [model) bf that system arid Its cbmponerits^ 
reiatioriships arid dyriamlcs In at least a reasbriable facsimile bf their 




byerall corn p lex! ty. the ievel of descrijStive detail needed in the jDdr- 
tralt is estabHshed by pqhcy needed for . a 

vaMd object of re^searcK needs. The extent to which 

the portrayal is suffiqiently complex to have e^xplanatory power thrbUgh 
the relations ^and dynamics captared is an index or tne dtility of tne 
model in deciding tjie* policy and research Issues raised by the ihstruc- 
tibhal delivery system. - ' 

The Second lEA Mathema^tics Sta^y was conceived in the arena of 
such issuesi It seeks to assist participating countries in their own 
mathematics ihstructibhal delivery systems, whether from a perspective 
of policy, research or pract[ce, arid it also seeks to provide resources 
that will help those cbricerried with policy or research to more broadly 
explore issues irivblvirig cbmparisoris bf alternative systems. 

The First lEA Mathematics Study was cbriceived in a similar areria 
and faced a similar task of pbrtrayirig matJiernaties iristructibrial delivery 
systiftis for a group bf cbuhtries pi^in^^'Iily for pblicy fDurfDbses. That 
study chose to fbcUs its'* descriptive jDbrtraft bri a careful picture of the 
system's outputs of student achieyerneht and to focus its explariatbry 
efforts on key background and ihstrUctibhal variables. 

i_ The Second IE A Mathematics Study, building on the work of that 
prior study, seeks to erihance the usefulness of results both iriterria- 
i^PP^UY 3ild _to_each of the participating countries in terms of their own 
nationaL concerns. : It seeks to enhance' the usefulness of results for 
those wtio approach instructional delivery systems from a policy per- 
spective as well as for thdsis with concerns for research arid pr-actiee. 

jt is^esirabie arid useful ^ provide a clear and detailed portrait 
bf the outputs of the various national systerns, i-^e. ,^ the actual achieve- 
merit- arid attitudes bf s^tuderits^ _ However, it is also important to 
greatly eriharice the pbrt^rait^of TOthematJcs instructional de[ivery sys- 
tems, tb move beyond the factors identified by the First Study. To- 
ward this erid, the Secbrid Study cbriceives of an instructional delivery 
system as irivblvirig three key elemerits, as follows:- 

1. The System Specialists , who form riatlbrial educational inteji- 
tibris into a curriculum iembbdied varibusly) , trairi fDersbrinel 
arid distribute resources for implemeritirig this intended cur- 
riculum^ and prbvide mechanisms for mbriitbririg arid guidirig 
that implementatibri; 

2. , Tjm4fistr4JGtional Agents , the teachers who prbvide the 

instruction that implements the intended curriculum iri the 
schools and classrooms, 

3. The Instr u ctional Recipi ents , the ^ students who receive the 
curricular impujse started-^by the intentibhs dealt with by the 
System Specialists and tr;ansmitted by the teachers as instruc- 
tional agents by whjch the jm^pulse^^^t^^^ the de- 
livery system from originators to recipients. 



This . cdriceptualizatidri of ihtrUctibhal delivery systems leads to 
three levels of focus for the Sepdnd Study: the e<lUcatibhaI system 
(geheratjng the Intended Cur ric^.UJi^) ^ schools, classrooms arid tjeachers 
(gerierating the Implemehted Curriculum}, and students (embodyihg the 
Attained Curriculum), these thrife^ levels of fdcUs lead td the three 
key components of the Second Study^-describetf>^ 

Tl^is jriujti-levisj conceptUalizatidn offers the. adviHt-a^^-^fSprdv 

detailed portraits of two aspects of Instructional deLiv^y^s^tW^s (the 
Intended and I mpjementisd Curricylums) not^potyi:Bfed in subn^i^ail be- 
fore. It also offers the advantage of dettnefung policy and research 

quest[ons.^abqut dynamics and relations between inputs (the Intended 
Curriculum) ar^d outputs (the Attained Curriculum) into two sets df 
related questions using the Implemented Curr 

yehing variable. That is^ policy and research questions of interest may 
be explored by^ considerjng fij^st the impact of tjie Intended Curriculum 
oh the Implemented Curricalam and then the [mpact of the^ Irrplemente^^ 
Curriculum on the Attained Carriculum. Thus, seems probable that 
both the descciptive and explanatory aspects of the portrayal of mathe- 
matics instructional delivery systems will Jbe enhanced by the multi-level 
conceptual izatidh of the Second Study, both between and within coun- 
tries, and for policy , Research and practice perspectives as well. 

The irivestigatidri will include: » \ 

^1.1 An analysis df the mathematics cUrricUjUm in order td determine 
the nature of the curriculum today td specify the changes which have 
taken place in the cUrricUIUm in the past tvvehty years, and to identify 
those factors which have contributed td its current character. 

Benefits of the Curriculum Analysis 

For most lEA countries, the past twenty years have been a period 
of considerabie activit^y in cprrjculum development.^ W the 
^^suUs of this enormous^ investment of time,' talent and financial re- 
For example, has the curriculum become much more hetero- 
geneous, signifying more responsiveness national goals and needs? 
Or, in spite of (or, as the results of) the activity^ and effoj:ts, does 
school mathematics appear across countries, as essentially a monolithic 
structure? , , 

Detailed ihformatibh about the curriculum can also be of irtipdrtihde 
in viewing the findings of the other two phases of the study: the 
investigation of ihstrUctfdrial plr_actice and the analysis of student btit- 
ddmes. "CUrricUIUrri, " as GriffLths and Hdwsbh have noted [London: 
Cambridge University Press, 197^, page 156) involves not only state- 
ments^ abdUt gdals and cdhteht, but includes either explicit or implicit 
notions of pedagdgical methdd and df evaluatidh. Hence, a khdwiedge 
of the cirriculum df a cdUhtry shdUld help shed light Upon teaching : 
rnethpds utilized to implement the content of the curricfulum and should^ 
also be of assistance in understanding student dutcdmes as measured by 
the International tests and attitude scales. 



16 



13 



The curricuium ahaiysiSi ' therefor^ purposes which in 

some senses are indoperideht of each other. It provides much hgided 
infdrmatidn about the currlcuium in each country within the. cdhtext_ of 
knowledge about tHe cUrridUlUm across some two dozen countries. The 
curriculum analysis also serves to help Understand arid interpret the 
data to be collected at the classroom level {both teacher arid pUpil). 

it. 2 Ah investigation of classroom processes tPdpuIatidil A) 

A series of unique, detailed questionnaires has been devised' for 
the jDUrpose of obtaining jnformatid^ they teach 

selected topics Jn the Pof^ulatjon A curriqu 

chosen on the basics of an international consensus on what is important 
subject matter for that Popuiation. 

Benefits of the classroom. process study 

Very little detailed irlfdrmatidrio is available dri what iristrUctibrial 
strategies are employed by teachers as they go about teaching. Yet, 
since the classroom is the heart of th^ educational process (at least as 
educatibh is most cbmnTohly practiced today}, it is essential that we 
have more ihformatibh abbUt what students; encbUhter as they study in 
the mathematics classrbbrri* - ; 

TRis infdrmatidn is alsd heeded ^s the pre-siervice ahd [h-service . 
heeds of teachers ih a cbUhtry are assessed. What aspects bf teaching 
practice seem to be most commdh? What are the desirable, aspects of 
this practice, judged dh the basii bf cUrreht prdfessibhal wisdbm about 
pedagdgy? What aspects df ihStrUdtidhal practice' heed td be imprdved? ' 
For example, do teachers use a variety df fhstructidhal tethhiqu^s, dr, . 
instead, do they' tend to have an approach which 1s applied to all 
top[cs, and to students of alL^ prd- 
fiies" of teaching behavior which caii characterize a country? 

An attempt, wijl a|sq be made to r^ practice to 

student learning. What categories of^jhstruqjional practke are related; 
to student achievement, where this achieyejrient Js a measure of growth 
on a particular topic In that teacher's classroom during the school year? 

4.3 Ah arialysis 6f student outcomes 

Achievement and. attitude instruments have been devised to reflect 
emphases and cbhcerhs in mathematics educatibh for. Ah interriatiorial 
item pbbL bf several hundred items for each pbpUlatibh has been devel- - 
dped. Drafts bf the ihstrUrriehts were prbpbsed. by the Internatiohal 
Mathematics Cdmmittee, ahd react idhs ihyi ted frdrn_ the Natibhal , Mathe- 
matics Cbmmittee ih each participating country . The cognitive ihstrti- 
mehts will prdvide ; ihforrnatidh at the item arid sUbscdre level bh mathe^ . 
matical cdhteht ahd behayidral prdcess dimehsidns_ bf achievement base^ 
dh ah jhterhatidhal grid for each pdp (see Tabjes 12.1 arid 12.2J. 

The attitude Instruments will measure, these aspects df mathematicd- 
related affect: mathematics in school; , mathematics and self; mathematics 
3S a process; mathematics and s^ scale devised td measure 

student attitude toward computers. 

o 
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Benefits of the student butcbme tbmpbheri 

This aspect of the Study will enable ^ the level of rr^thematical ; 
achievement to be assessed in the light of curricUlar emphases arid 
instrtictibhal practice. What is the level of computatidrial skills in each 
cbdhtryl: To what extent are students able to solve mathematical, prob- 
lems? What are the attitudes of stUderits about: studying mathematics, 
arid abblit its role iri society? How db comparable ability groups (say, 
the top 5% of the stUderits) compare iri problenr^sblyirig skills across 
countries? 

In plaririirig for this Study, the IMC has emphasized the importance 
. of utilizing information from the research instruments at the 
Two reports, by Peaker and by Postiethwaite, illustrate the sort of 
analyses which are expected to be useful. 

F?eaker ^inte rnational Review of Edacati oa , Volume XV, 1969, pages 
222^228) examined the- findmgs of the FJrstjMathematics Study from the - 
perspective of _3leaning_ those results of greatest [hterest to cjas^room 
teachers.^of 'mathematics. Hi^_ analyses^ of item level irifbrmatlbn ar^, 
partieularly rich in jmpjications foj- instradtlori. For example, he exam- 
ines country performance on ah integration item for PbpuJatibri 3a (ap- 
proximately our Popalatjon B), [Test 9, Item 4] arid coriclUdes that the 
striking differences in performance bh this item (.04 in the Neth^rlahds 
to .68 in England) canhbt be ascribed cb the reteritivity of the system 
(which is 5% for bbth epUntriesJ but shbuld, rather, be attributed to ; 
differences In curriculum arid instructibri. An arialysis j)f perfbrmarice 
at' the sUbsebre level produces similar cbricjusib While twb cbUritries 
may have total scores which are riearly ideritical> the contributioris to 
this scbre come from different sburces. Agairi, iri Pbpulatibri 3a, Bel- 
gium arid _ErigIarid had total scbres; of 65 and 66 respectively. But, 
observes Peaker> ''Belgium is strbrig dri new mathematics, algebra^ 
arialysis arid sets. Englarid is Itrbrig on gedmetry>' arialytical geometry 
arid particularly bri calculus. " (page 225). ^ 

. Pbstlethwaite E'Mtem scores as' feedback to Gurriculum planriers: A 
simple case from ' the Swedish Comprehens[ye School arid a more general ^ 
model; 'V : Scandinavian J&urna^ of . EcUic ational Resear c h (15:3, pp. 
123^136)] has suggested that item level ajialyses in cp^^^ 
examination of the textbooks, and syllabi may yield important information 
for helping explain variation in achievement at the educational system 
• level .- . . , 

RaljDh W. Ty[er, In a recerit paper erititled "Amerlciari Education in 
the perspective of educatiori Iri other riatloris", reviewed the state of 
eduj:atign , in the U.S. on the basis of the previous lEA studies. Mariy 
of his cbhclusibris may be of interest to thbse iri all cbUritries who are 
concerned with the education of their youth. He viewed the lEA data 
from the ppirit of view of "ideritifyirig factors that explairi the variarice 
iri educatlbhal achlevemerit both withiri arid among iridUstrialized ria- 
tibris." He goes ori to cbrielude, "For the top five or teri percerit of 
yburig people in all these riatibris, the twb mairi factors are oppo'^tunity 
to learn the subject arid the emphasis giyeri the_sUbject by those adults 
who are respected by these ybuhg people. Fbr the majority of the 
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children the factors identified are the bppdrtunity to iearn the subject, 
the edUcatibri of the parents, the p the subject and . 

the time devoted to learning it.'! . 

4.4 Ah examiriatibn of changes in mathematics education since the early 

... jgiBos. . • • • 

The past two decades have been marked by considerable interest 
arid activity' iri the curriculum; particularly in matheniatics_^^^ 
sciences. Iri mahy cbUritries> the 1960s espeGiaUy were characterized as 

a period of curricular relbrm, and even, reyolution. Jwo events afDprox- . 

imately twenty . years ago now serve as im.portant benchmarks In mathe- 
matics education. Jn 1958, the Organ izatjon for European Econo/nic 
Cooperatipn conducted a survey of matli^matics edacatibri in member 
countries which culminated in the Royaumont ' Seminar and the refDort^ 
New Thinking In School^ ^aath^gmatics (OEEC, I96l). This survey pro- 
vtded i n fq rma t ion o n^ pr act ices an d^ t rends in sehbbi mathematics arid 
rather detailed information on the^athematlcal^ content bf the curricu- 
lum. The second event, the First lEA Study of Mathematics, 1964, 
provided empirical data bri characteristics bf ischbbls^ teachers arid 
students as well as achievement arid attitude data for stUderits iri tw 
countries. 0f those iwelve; ereveri are currently ^plaririirig. for partic^^ 
patibri iri the Secbnd Study. ' ■ - 

With iriformatibri from these two surveys^ sUppiemented by informa- 
tibri curreritly being - requested bf the some twerity-four cdUritrie^^^^ 
have expressed interest iri the Second Study, it wijj be possible to 
chroriicle mariy bf the chariges which have occurred in mathefn3tics 
educatibri. . i 

Typical questibriS : which cduld^ b^^ 
changes in mathematics education includer What lasting^ chang in the 
curriculum have' taken ; place? What factors^ have [nfiuented^ t^^ 
chariges? How does curricular change tak^ p|ace in different countries? 
With respect to student outcomes, guestjons relating to cu 
for "declining standards" can be addressed.^ jn what aspects of mathe- 
matics achievement ha\/e t^he^re^ been declines a what aspects have 
there been gairis ovej the past twenty years? What evidence is there 
of the impact of technology, in particular^ cbmf^^^ arid ealetilatbrs, bri 
c[assroom practice and student achievement and attltud^^? Are there 
strikingly different patterris bf student gains (losses) across cdUritries? 
If so, what other faetbrs cari help accburit for theoe changes? 

General que'stlbris relatirig tb student acl^^^ as suggested by 

Tyler (bp cit) will be respbrided tb iri the study. "Wide pUbiicity rieeds 
to be given to the negative effects bf the erbsibri bf "the bUt-bf-schddl 
learriirig erivirbrimerit. The average child ndw has fewer hdurs with : 
parerits, arid fewer hours iri ybUth-servirig jristitutidris than in the 
past. The average child frpm ID tb 14 years of age sperids 1 >5_00__hd_Urs 
per year watchjrig televisidri arid drily 1 >1 00 hours per year in school. 
Herice, rebUildirig the eroded but-df-schddl learning: envirbriment^ is 
likely tb bririg greater results than any dther single strategy." 
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5. Model for the Study ■ ^ 

Figure 1 illustrates a conceptuaMzat[on of the Sti^^ Thre§ levels 
are objects of investigation^ ^ one corresponding^ the three 

cbmpbhehts of the study. The curriculum analysis' focus^^^ 
educational system as an entity and is intended to portray the cp^^^ 
in which mathematics education takes place (schoo[ orgaaizatiohi sfelec- 
tivity and educatipnal goals, for /example) as well as "to review the 
status of mathematics leducati^^ The investigation of 

ihstructionaj practice focuses upon the classroom level. /Here is deter- 
mined both the extent to which the '"[n^tended carrlcul^ 
ated at the . system JeveL^ Is act u^^^ !h the tjassrdom, and 

the variety of instructional methocJs employed as the implemehtatipn 
takes place. The third focus Is upon student attaihmeht of cuhricUlar 
and Instructlpnia! goals. What is the nature and extent of mathematical 
achieyement and attitudes of the students in the two target populatfons 
of the Study? 

An Expanded Model of the Study - 

Each of the compbhehts of the Stud may be viewed in more detail 
with the aid of the expanded model (originally proposed by MqKnightL 
as shown in Figure 2. The figure is useful in highlighting certain of 
the interrelationships which are tb be*examined. 
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6. Cdrtlpdneht i: CUrricuium Analysis (The intended Xurriculum) 

. The cjurriculum analysis is to be conducted in order to prpvide a 
backdrop for viewing and interpreting the findings, of the ilassroom 
processes and student outcomes cbmjDbriehts of the Study. The cur- 
riculum analysis will Uhdertake two jDrobes into the state of mathematics 
curricula in the participating countries and brgahizatiphaJ contexts 
surrpunding these curricula. These 'twd probes > discussed below ^ are 
referred to as a Cdritexts Survey and a Status Surrey. 

6.1. Xbe Contexts Su rvey^ 

[t is Jhe gdal of this cdmpdrieht of the curriculum analysis to pro- 
vide a framework within which the findings of the status study can be 
viewed; Some of the data on the contexts for school mathematics is 
^already availabJe in such^ i^^^ as_ the RdyaUmdnt Study and the 

First lEA Mathematics Study. These d^ta wiM be updated where pos- 
sible, and fresh dsta will also be sought. Explicit probes will be made 
in the following areas: 



. 6;1.1. Societal contexts for schooling w reference 
the occupational structure and national demography. 

611.2. Iristitutidhal edhtexts of schdol mathematics: 

6.1.2i1 Articulat^ldn between^^s school and the 

- : subsequent careers of students. * 
6.1.2.2. ehrollmerit, tracking and sehddl drgahlzatlon 
. and overall eurriculum witH .'"^e implicatidhs for 

• , cotirse erirollmerits and perceptldhs of mathe- 

_ _ _ _ niatics; 

6.1.3. The curriculum deveropmerit system. 

\lts character and form and its particijDahts at the formal 
~ Iwel^ and its articUlatidri with a cUrricUlUm research af\d 

develdprnent system, if ariy. ^ , - ; 

6.1.4. The characti3r and fdrm of the cdritPQ^ 
coprdinatron surrdunding mathematics^ i.e., ^ 

6.1.4.1. examination and grading systems . 

6.1.4.2. inspection systems 

6.1.4.3. text and materials development systems 

• . '_ " ■ '.^ _ _ _._ 1 . . . . _ ■ . _ _ * 

Particdar attentj^^ jurisdictional settings of 

these systems and their articulation with the formal structures of schddl 
drgahization; 

6.1.5 The legal and certification structure of the human re- 
source system: supply and demand of teachers.. 

A questiohhaire has been devised by the Curriculum Analysis 
'Group In order to formulate a description of the societal and drgahiza- 
tibrial contexts of mathematics education with a fodus on occupational 
structure and hatidhal demography, articulation between secdhdary/ 
schdol and subsequent careers df students, schdol drgariizatidh/ cur/- 
ricUlUm developmeht systems^ cdhtrdi and cddrdihatlhg systems (exami- 
natidns, inspectdrates, etc. j jnd the legal and credehtialihg\ structures 
of the system. . / 
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In J959, the^ Srgahization for^ Ea ropea h Ecoriomk: Gbb^^^^^ 
(GEEej hejd a seminar in Royaqmont, Franc^ re- 
viewing the Gurj^ent st?itus of the school ^rrtathema tics currica!am for the 
21 member coantrles of OcEE and to consider the "prodactloh of a 
sound mathematics prograrfime !n harmdhy with modern thirikihg In 
: mathematics," ( New Thinking in School Mathematics , OEEe> 1961, 
page ?•) . o ' 

^ The_earr|ctj!'am Analysis Grbap has prepared a qaestibhhaire based 
bri the Rbyaumbht sUrvey ihstrumeht, but mbdified in order to take 
accbuht bf mathematics tbpics which h^ve since been "legitimized" in the 
curriculum arid hew tbpics, such as cbmptiter br irifbrmatJbri scierice 
which are candidates for "legitimacy" > to varying degrees^ in the cur- 
ricUlums of the lEA cburitries. 

There ' is signiTicant oyeHap between cpuhtries participating in the 
cUrreriL Second lEA Mathematics Study arid those participating iri the 
OEEC Rpyaumprit Study (where possible^ a picture of the curriculum as 
it was in 1960 will be cpilectetJ from those lEA countries which were hot 
. members of the OEEC at the time of the survey. The countries partici- 
pating in the Royaumo^nt Studx _were:_ Austria^ BelgiiMt Cjin 

Denmark, France, Germany, Greece^ Ireja^nd, italy, Luxembourg, The 
Netherjands, Norway , Sweden , Switzerland, Turkey, United^ K 
United States, Yu^qsl^^^ ; ) The expectation is that the census- 
like surveys wi!| pj"ovide; for a significant n lumber of countries^ us efu^^ 
arid [mportant data on both stabi^lity an^ chajige in^ the mathematics 
curricula ^ in the participating countries. There have been major 
changes, iri the strdctares bf mariy schbb! systems sirite 1960 arid it will 
be of cbrisiderable Impbrtarice tb attempt tb assess the sigriificarice o? 
these changes for the charact^ bf the mathematics taught iri the 
schbbls. . . 

Oi3 Natibrial Case Studies 

' Iri 1977 ^ari advisory cbmmittee of UNESCO/ spprisored by the 
Interriatibrial Commissipri bri Mathematics IristrUctibri (ICMI J , issued a 
call ltd various countries for reports on changes_in mathematics edu- 
catibri which have bccUred since the late 1950s, The call iricjuded the 
follbwirig butlirie, published in EdUcjtjpnal Studies iri Mathematics ^ tp be 
used as a guide to the authors of the national reports as they des- 
cribed the various deveioprrients in their countries: 

Chariges in: 

subject matter 

^ teaching method . 

a ttjtude toward ma^^^ 

relations between mathematics and other' subjects. 

0 Change as a process:^ ^; • 

The leadln^^ldleas -- _ ^_ 

how did they develop in the course of the years? 

■ 24 
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how did they manifest themselves in plans, projects, 
proposals? 

how were they actually realized? 
The mam forces ~ - ' .* " 

general educationai, s cultural developments; 

increasing mathematical literacy; 
increasing mathematical excellehce; ^ 
adalt ieducation; 

applicability of mathematics. 
The designers''— : 

cbmmissibng, committees, curriculum developers, textbook 
. - writers, test designers. - 

The performers — 

_ admimstrations, schools, teachers, students, parents. 
The tools — . ■ ' 

training,, retraining, textbooks, syllabi, teacher's mah^ 
uals. \ c 
Cddperatipn and resistance — ^ . : 

by admihistratibris, teachers, parents, public. 
Lessons learned in the past to the benefit of the future/ 

Changes on different jivels : 

sch(C5ol, teacher training and retraining, and guidance In their 
inter-relatedhess. 

ieactiphs to^ change: ^ 
in other areas; 
by higher learning; 

sby parents; * y - . ' 

3^ the public. 

RepbrtsNwere recejved from the following countries (authors of the 
rejpbrts are giVen in parentheses) and pubjished in Educational Studies 
m Mathematics A Volume 9, Numbers 2^ and 3. Co untries planning to 
parUcipate in the Second lEA Mathematics Study are marked by *. 

Australia* (Ak^^ Blakers) Nigeria* (R.O. Ohuche) 

Bangladesh (S.>M. Sharfuddin) , Poland (A. EhrehfeUcht) \ 

France* {A. Re^z} . , Si^erra Leone (A. Williams) 

Great Brltaih* (AuC Hbwson) Sri .tanka* (A.J. Guhawardeha) 

Hungary* (Maria Halmos & Sudan CM. El Sawi) 

r^'-^FfM^^^ Thailand* (0. Purakam) 

Indja (J.N. Kapur)^ , United Stated* (J.T. Fey) 

Iran (Badlollah Rostamij West Indies (B.J. Wilson) 

- The Netherlands* (H. Freudenthal) 

_ __ A^lbsequently, reports from an additibhal .half dozen countries were 
received and are scheduled to be published by bNESCO in Summer 

I «l / zl • ' ^ — — — 

"'' _ 1 1_ .■,','»■_■.. " ( • ■ 

^^ ■^These reports wijl be examined by a icdrtimfttee- of three persons 
r'i^^^^rl** der Blij^ Hildirig, Sweden; and 

A.I. Weinzweig^ the On|ted States) which will prepare a summary paper 
synthesizing what is reported by the countries. Professor van der Blij 
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is serving as chairman of this cornmittee. Robert Garden also assisted 
greatly in the work of the committee; 

^5.4 Case Studies in Teaching Mathematics ' « 

the final component of the carr|cajam analysis will deal with three 
mathematical topics and how these topics^^re articaiated in the curricu- 
lum and classroom. For Hlastrati^ve purposes^ three countries W^lkb^ 
asked to prepare national case studies oh the teaching of one /of the 
three topics: geometry^ Introductory algebra^ and" probability arid 
statistics. . 

6.5 Publications of the eurricalum Analysis 

Th^ following pubMcatJbris^ scheduled tb/appear in mid-1 are 
eX|Dected to emerge frq^ the symposium^ arid hertce be the first reports 
of the findings of the Second I riterriatibrial Mathematics ^Study 

6.6;1 Report of the prbceedirigs of the Symp)Sium 

616.2 National reports ori tHS status of mathematics education 
arid the synthesis report 

616.3 ImplicatLbris of ^#he curriculum analysis for jnterpreiir^^ 
the firidirigs of thi classroom process and student oat- 
comes jDhases of the Study 
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7. Component M: Classroom Processes (The Implemented Currictiltim) 

Previous lEA stadies have' soaght to provide detailed portraits of 
student achieveme^ represented the curricuja ibf particijDatirig 

countries as attained by the* students in the targeted fDopulatibriSi 
These detailed portraits of national student achievement, set iri ah 
mternational context of portraits^ of student attainment in other cbuh- 
tries; had a variety of policy implicatiohs fer the jDarticipatihg^ cbuh- 
triesi The [Dbteritial policy imjDiicatibris were enhanced by studies of a 
number of broad baekgrbuhd variables ebhcerriihg the students involved 
and a few key gibbal eharacteristicis bf the ihstructibhal delivery sys- 
tems bf participating countries. 

7il- Descriptibh and Explahajtibh 

The gbal bf the Classroom. Processes cdmpohent of the Study is to 
prbvide detailed descriptlbhs bf hatibhal curricula as implemehted in the . 
schools and classrooms of the I EA countries. The practicai complexities 
of school and classroom life lead to a trarisfbrmatidh of any intended 
curriculum (such as those portrayed in the Curriculum Analysis com- 
pdheht of the Study). It is pdssibie to Judge something of the extent 
and nature df this transfdrmatidn by examining those processes and 
activities characteristic bf ihstructibri in the targeted cjassrodtti^ 
by examining the methods and practices commoniy used By the teachers 
prdylding instructidn to the target classes. 

7.1 .1 betaijed ^ bescriptions of Practice, Providmg a portrait^ o^ 
classrooms involves, at the most fundamentar leve[, a detai[ed^descrip- 
tidn of typical practice in the classrooms in^ which jnstruction is de- 
livered to the target population. Such description is prior to any study 
of relationships amon^ aspects of that classroom practice^. However, 
eyen descriptive studies at this level should begin to provide some 
information relevant" to the "why's" bf student butebmes. Withbut the 

Jnformation-^ppovided-by -s 
which student outcomes must be explained. 

7.1.2 Relational DeseriptibhSc Tb enhance the explanatory pbtehtial 
(and policy implicatiohs) bf the classrbbm prbcesses descriptibris, it is^ 

— nec e ssd i y lb gu be yond the simple portrayal of discrete — ctas'srddm~ 
practice alemehts tb a portrayal, of the relatibnships between these 
eJememts. Ah Jmpbrtaht aspe bf the data ahalyses^made pbssible^^ 
the Classrbbm Processes cdmpdheht- df the Second Study is the examina- 
tibh of key iriter relationships between the classrddm practice elements 
eapturetf by the ihstrumerits used. What should emerge is a description 
both df discrete elements df classrddm practices and df interrelatidnshlps 
in classrddm practices that make more global characterizations of those 
jDhactiees pdssibie. 

7*1.3. Explanatory Analyses. The .curriculum as implemented in 

actual schddls and_ cjassrooms _ not q^nij^^ as context for 

interprettfMg^tudent outcomes and seeking reasons for them, out is also 
impcrtant as a source of potential explanatory variables. Classroom 
' ef^cts And teacJier^ essential intej^vening yariabjes in 

describing any national instructional delivery system. In most cases, it 
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is not realistic nor should it be particularly effective to distant 
backgrburid variables arid variables related to the dfficiar or Intended 
curriculum of a country in order tQ_ explain the pattern of student 
achievement and _ attitude butcomes . The intended curriculum an d :the 
datibhal system for directing the implemehtation of that curriculum must 
first have ah impact bri the schools and ciassrooms which are the s^^ 
instructibhal delivery. In return^ this deliyered instruction has an 
impact bri actual student achievements and attitudes^ Thus^ classroom 
effects arici teacher practice must be^a central link in any detaijed 
explariatbry chairi seekirig to characterize a national instructional de- 
livery system. 

This status as source of L^ter^ that the data 

from the Classroom Processes comgD^nent will enter ihtb two other impbr- 
tant kinds of analyses. First, classroom and teacher practices char- 
acterizing the imjg^iemerited curriculum may be-^ related to the^ intended 
curriculum and the national system^ for guiding implementatibri (both 
portrayed In the C^urrlcujum^ Analysis cojnpoheht of the Study! tb assess 
the extent and nature of transformation of the intended curriculum arid 
to assess how such tr^ns^^^ is related tb various kinds arid 

aspects of national systems for guiding imjDiemeritatibri. 

Furthermbre, aspects of the implemeritekJ curricUrum may be related 
to the pattern bf student achievements arid attitudes. The design of 
the Study seeks tb make such ari arialysis jDossible bbth withiri several 
specific mathematieaJ topics arid acrbss those topics. It seeks also td 
analyze several differerit levels bf classrobm practice variables ranging 
from the very specific tb the mbre igerieral. This enhances the likeji- 
hood that this type bf analysis^ at some level of specif idty/ maj^ re^^ 
irripbrtarit relationships that might dtherwise be hidden, by some more 
gibbal variable (such as "bppbrturiity to learn") in a less detailed 
arialysis. There is thus the potentiar for discovery of patterns of 
iristructibrial practice that have demonstrated effectiveness in terms of 
studerit achieyemerit arid attitude outcomes. 

7.m. ^ Summary. The Classroom Processes cbmpbneht of the_Secbnd _ 
Sturfy wili poffentiajiy be ab[e Jo pr^oj/id^e^^^a 

ins^uctio hal del iv ei:^_tba.t-includes the following descriptions and analy- 
ses: 

i) bescriptibns of djscrete 

li) Descriptions of relat|qns bet^een^ c|assrobm practice eleraerits 
(indudjng mq^re globaK practice characterizatioris J 

. Hi) Analyses of relationships between interided arid imjDlemerit^ 
curricuja (and effect of various practices in guidirig imple- 
mentation) J' 

ivj Analyses of relatibriships betweeri iristructibrial practice as- 
pects of the tmplemerited curricurum arid the patterri of stu- 
dent achievement and attitude butcbmes 

rfT2--reategbrtes^^^^th[^~°ir^^ : '. ^ 

Section 5 of this Bulletin bffered a model for this Study and Fi- 
gure 5. 2, 'is ari exparided versibri bf that rtlbdel. The ceritral "blbck" of 

..' : . . :g§^., 
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that figAJre represents ah analysis of the Implemehted Curnculum in 
terms of four ma|6r categories^ arid seeks to fDut these four categories 
into a network of categories characterizirig the other cbmpbherits of the 
Study. 

7.2.1 • teacher Methods and Practices, Cehtrai to the impiemerited 
curriculum > to iristructibri as actually delivered in the classrbbm, is the 
category labeled "teacher knowledge: methods arid practices." Iridi- 
vidual teachers often act, when faced with similar iristructidrial situa- 
tion^, in stereotypical ways that constitute a pattern or set of patterns 
characteristic of their teaching practice. Iri some cases for a teacher > 
these jsatterns are a refiectiori of cdnscidusly held and deliberately 
executed methods, jn other cases, these patterns have evdlved out df 
the teacher's exjDerience past practice. The result is, fdr any 

given mathematkaj^t^ array or reper- 

toire of characteristic approaches to instruction in the varipus aspects 
of this topic. These j^ractice patterns are quite complex but , jn many 
ca^ses,^ are relatively stable and ayajlabje to the teacher for considera- 
tion (at least if probed by very specific questions in which demands for 
inferences and demands on memory are minimized j . 

A model has been ^cbristructed arid Is beirig refiried which relates 
some cf the major aspects of such teacher methods arid practjces^, apd 
which also iritegrates them with the other aspects of the implemented 
curriculum. A series of ques^bririaires for teachers has also beeri 
developed which probe (by mearis of very specific low-irifererice, low- 
merridry qtiestidris) critical aspects of these kirids of practices iri several 
tbpic areas. These questibririaires, albrig witin the model, shbuld pro- 
vide both useful deserijDtiqris of discrete practice elemerits (as discussed 
earlier) arid descriptibris of relatibris betweeri practice eleraerits leading 
to more, global characterizatibris bf teacher practice (guided by the 
model). 

7.2.2. Other Aspects. While "teacher methdds arid practices" are 
. the ceritral cdmpbrierit bf the mbdel of the impiemerited curricurum they 
are ridt the dniy cdmpdhent. There are several categdries that impact 
dn teacher methdds and practices^ t^^^ cbri'strairi arid iriflUerice the 

serectidri df™methdds7~arid"^*alsd~td affect the executidri dr perfdrmance df 
a given typical method dr practice. One such majdr categdry is that of 
"schdoPand classroom resources." The array of resources available in 
the schddl or a ejassrdom within the schddl cdnstrains the instructidrial 
choices of the teacher proyidjng instruction in that setting. Respurces 
act as both props and cues in instruction. Certain instructional activi- 
ties can be carried out on[y if certain resources are ayailable as props 
^9_^^_"sed in^ the activities. __Further, the actual ph^sjcal presence pf 
certa^in resources may serve to cue or influence a teacher's dedsion to 
use certain activities. 

A second major category is that of "teacher beliefs and attitudes." 
This category includes a variety of factors internal to the teacher that 
. iriflueriee his or her iriterfDretatiori bf Instractlon^l^ situatioris - arid re-^ 
spdrises to^ them (in terms of methods selected arid executed) . These 
fectors include such thirigs as teacher beliefs or schemas of typical 
studerit behaviors i iristructibrial situatibris, arid teacher aetibris. They 
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aiso Incldde the teacher's ihterhal representation of the mathematic^al 
subject matter; and basic attitudes toward mathematics, towards teach^ 
Ihq and towards specific classes and students.- Teacher values relaj- 
nl'to various kinds of activities, student responses, are also include^. 
While this category is ah incredibly complex one, certain aspects ^ith 
demonstrated connections to practicaj instructiorial outcomes have been 
selected for probing through the questionnaires. 

TWo other major categories that directly affect the implemented 
curriculum and, in particular, teacher method^and practices are^^^^^^ 
elements of the iritehded curriculum.^ This includes, first. _ the textbook 
6r textbooks chosen arid available to the teacFier. This is one of the 
dominant influences, (if not the dominant influence) teacher choice of 
instructional activities and "itrategies. Secondly, there are^ various 
embodiments of curriculum goals and influences^ such n =ff"i5 tiS^ 
school syllabi, external examinations, etc., ^hich affect the _activ. ties 
and strategies chosen not so much in the sense of by what they make 
possible (as a textbook does) but in the s|hse of by what they make 
desirable (which a textbook also does). Selected asjaects related to 
both of these categories are also probed by the questionnaires. 

Finally, there are the actual activities of the classroom. The 
previous set of four categories affects and constrains the selection and 
.Execution of teacher methods and practices, arid the methods and _ prac- 
tices so selected are realized in a set of instructionaL activities jn (^r 
related to) the classroom which constitutes the actual instruction for the 
target class. 

7.3 description of the Instruments 

7 31 Instruments Involved. Instruments or elements of ihstruments 
related to the Classroom Processes component of the Study include the 
foiiowjngr » , 

1) Teacher Background Questionnaire. ^ . _ 

This seeks iriformatLon on sex, age, years of exjDerience, etc. ■ 

V 2) Teacher Attitude Scales .. ^ , ^ sii 

Certain of the attitude scales [described elsewhere] will be 
taken by teachers as well as students. 

3), Teacher bpportun|ty-to-bearh Items _ u su 
. This coTiStruct was devised for use in the First J EA , Mathe- 
matics Study as an index of the extent to which the jntended 
curriculum was implemented by the teacher in the cLassroom; 
The measure has subsequently been refined and is currently 
beinq piloted for. use as one aspect of the Classroom Process 
comtxinent of the Second Study. It is planned that both 
teachers and students will be asked questions concerning the 
extent to which opportuhity has been provided in class to 
learn the various topics reflected in the items from the cog- 
nitive testSi ^ .; . 

i») Classroom Process General Questionnaire 

Certain aspects of teacher and instructional practice are typi- 
cally quite uniform across specific subject matter topics, a 
general questionnaire will be used, seeking information on 
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'■ these more gene practice elemehts (e.g.^ grdUpirig prac- 

tices, uses of ihstrUctibhal ifiaterials) . Items from thi^ ques- 
tibhhaire, serve three purposes;: . _ , 

(13 They le^d to vahabjes poWe^^^ in their own right; 

(2) They _f3rpvide. a context for. in more top ic- 
' specific ihfdrmatidhj and 

(3) They make contact wjth variables used in other 
' studies of general teaching behavior so that results 

of . thjs study ma be related to other studies. 
5) Six Tbpic-Spedfic Classroom Que^tibnnaires' , 

Certain other aspects of , teacher and mstructiona| practice 
may or may not generalize across topics, but, even if thqy 
do, may still be^ best assessed through |qw Jnference jar[d 
memory demandin^gV^ specific aspects of instruc- 
tion related to specific topics. : '__ _ . 

A tpplc-specjfic questjqnna[re about J practjce has 

been j^deyelpped for each of/^six topics which are reported as 
["^P^C^^r^ o'^^'^^ry ^Jn^poC^silt _^ ^sis across 

cquntj^i^s. Thus, at least some of these Instruments should 
be related to the year's work of any target . class of any 
parti^patl^g ^co^n^trv^^^ Ihtehded that ^ach mstrdment_ 

yvHl be compieted soon after the majbrity of the year's instruc- 
tion in the relevant topic. . 
The six topics are as follows: 

(1 ^ Ratio, Prbpbrilbri ahd_Perceht . 

(2) . Cbmmbh arid Decimal Fractibris 

(3) Gepmetry 

iU) Measurement _ 

(5) Fbrmulas arid Equat^bris.; 

(6) Iritegers 

This array of iristrumehtatiqn should be suf^ to generate a detailed 

portrait of thej ihtehded curriculum, as implemehted by the teachers ih^ 
vblved iri the Study. 

7.3.2 Content of the Instruments. ' The six topic-specific question- 
naires, which cphstitute the^ instr^uments ^ qf _the 
Classroom Processes component of the Studyj are (designed to sample 
information frpm^ a rich . conceptual domajn (described in Sectij)n 7 . 2j . ; 
The humber and^ icpmplexlty of items scatter ed^ oyer the six question- 
naires jn^ightseern, at fi^^ Actually, however, all of 
the items can be groupepi into 1 3_itejn J:y^ and^ the Jtems of 
any gjyen ty^pe jlj^ follow, for the most part, the same item formiat with 
only minor, variations; arid (2) cluster together iri terms of the variables 
they represerit arid the higher-brder variables to which they cbritribute^ 

The 13 Item types fall iritb the follbwirig four majbr categbries: 

1) Teachlrig Methods (8 types) 

2) Topic Profile (3 types) 

3) Teacher Opiriibhs [1 type) 

,4) Specific Area Questibris [1 tyjDe) 
Two of the categories are qu^te simply described. First, at th^ erid bf 
each qtiestibririajre is embedded an "bpiriibririaire" that solicits a variety 
bf teacher bpiriibris bri aispects bf the iristrtictibri related tb the tbpic 
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ihyolved; Much of this information Is _pu rely is 
used . in characterizing teacher beHefs and values. Second, the "speci- 
fic area questions" cons^ute a smaQ set of J terns [7 in all) :which seek 
fbureiy descriptive information reja ted to instruction in one or the other 
of the six topics^, but which has no counterpart In the other topics 
te;g; / kind of geometry used) . , ^ 

The "topic profile" category involves three Itern^ types standardized 

across the six topics to provide a quick/ global profile of ihstructidh In 

that topic. The three item types deal with fl) allocated tirtie^ (2) 
sub-topics covered, and (3) jDrbgram emphases. 

The most cbmjDlex^ and certainly the most interesting^ category of 
item types is that of ''teaching * methods" which_ involves types. 
This category has five major sUb-categories, First, one suk5-cateaory 
relates to content dealing with "concepts and relationships." In this 
SJLJb-eatfegbry are two item types, pHe dealing, with kind of representa- 
tjbh used in the cbhcept-'Priehted instruction and another concerning 
the kind of teachihg techhiques used in introducing and dealing with* 
such conceptual represeritatidhs. , 

A secdnd major sub-category concerns "fbrma[as,^^_p^^ 
prdpdsitidns." It also inyoiyes two itenri types, with one sampjing the 
prdcedu res taught and a second the techniques used in teachihg those 
procedures. ^ 

" A third sub-category^ deals wi^ mathematical applleatibris. Again, 
two item types are Involved, one to sample the applicatibhs used and 
another to sample the sources from which these applications are drawn. 

A fourth sub-category involves only one item tyjDe Vvhich samples 
the reasons for use ^ahd non-use of certain qbhcept representatibhs, 
procedures, appjicationsf and techhiques frbrri the above sub-categbries. 
The_ fifth and final sub-category alsb involves one item type which 
relates to the number and kind of ihstruetibriar aids pr resburces uti- 
lized. ' " ' . 

This scheme bf prbfilihg tbjDics arid Iddkirig at teaching methdcls in 
terms of three kinds of eohtent tcbricepts^ jDrbcedUres arid applica- 
tibhs) , all bf which cari be realized through use of only about a dozen 
item types, seems quite simple. It is^ iri fdct^ this conceptual §impli-/^ 
city of design which allbws the gerieratipri of a reasonable but smaij 
riumber of higher-order variables to characterize the rich variety that 
can be sampled by the many specific items of the few item types; 

7.^1, Classroom Process Variables: An Overview - 

7.^.1 - Levels of Classroom Process Variabjes i_ 

The blassrodm process questionnaire will yield a wide variety of 

variables at several levels. At least five levels pf variables can be 

identified as fdilows: ^ 
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. . 1) ' Item level ; ^ ^ 

2) basic item-type variables (most- item variables may be aggre- 
gated with corresponding yariables for other items of the 

same item type to yield a variable characteristic of some 

aspect of that item type) . ^ 

3) homogeneous hig he rro^^ aggregates using 
conceptually-based clusterings of similar item types and 
variables) : J ^ - ^ i 

.4) heterogeneous higher-order variables (further aggregates 
using model-based cl.ti^ste^^ 5f morfe heterogeneous Item 
types and yariables ^hlch have be erV related through the 
modeling of teacher riiethod^ and practices) 
5) statistically-determined higher-order variables Caggregates of 
variables based oh stiatistiGal clustering tischriiqUes rather 
than any a pridri eoheepttial ajDpro^ch) 

, Item variables involve' rid ^clusteririg arid herice rid infererice in 
their defiriitipri. Basic item-type_ variables; irivdive miriirnal infererice 
reflecting drily the lowest level df crusteririg>_i.e. , that of identifying 
item types arid_ obvious cdritertt similar yield a 

large number of variables since there are a ' lifcg^ riumber df items aKcI 
^irtce this low l^vei of clustering yields dvef 50 basic item, type vari- 
ables (not cdnsidering the fact that most yield indices for . one or more 
df the specific topics a^ well as an dverajl. adrdSs-tdpic scor^ and also 
riot cdnsidering that^ in many ca^es^ a number df different weightirig 
schemes are availabie in aggregating by using frequency and/or empha- 
sis Informatldn gathehed by the instruments), c 

The next stages require some decisionsi, Two major approaches 
are available in seeking f^^ higher Inference cjustermg . One 

apj5roach sjrnpjy uses statist^^^ arialysisj to 

identify, weight and aggregate variable clusters. Conceptual interpre- 
tation of aggregates created in this_ way takes^plade jargejy after clus- 
tering, A seicond apFy"j)ach is Jq use a priori some conceptual scheme to 
identify clusters before any empirical work ^nd to bas^ aggregation on 
such already cdriceptually, mea^ningful clusters. - A conservative y^rsr n 
of this conceptual clustering inyolves stIM relatively low inference 
cjUsters 6f_variabies^my6lylrig similgr cdriten^. Thl^ !s^ what*j^ re- 
ferred to above as (relatively) hdmdgeriedus higher-order variables. A 
more verituresdme approach Is. to use a mdr^ elaborate cdhceptual scheme 
(e.g., a model that would generate profiles . arrid rig sets df variables, 
with the profile becdihihg a new higher-order variable) to generate what 
were called abdve (relatively ^ h^^terdgeriedus higher-drder vari- 

ables. • /■ o 

In the present sitUatidri pf^^^ infdrmatidn>; it ddes 

not seem reasonable to chdpge the statistical "rather thari the cdhceptual 
apprdach td higher-drder clustering or to chddse the more cdriservative 
rather than, the mdre venturesome in cbncfptual clustering, f hus^ the 
earl jer list reflects the fact that all .three strategies of generating use- 
ful highef-drder variables will be fdl lowed at lea^t thrdUgh the stage df 
large scale pi jbting. . It Is hoped that a convergence between statistic- 
ally -generated ^an^ generated clusters . will emerge during 
analysis of such data. . ^ ; 
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It Is; of course, not possible to describe at this time clusters that 
may be s^tat[sticaUy generated later. Work on model building and the 
Siore venturesome approach to conceptual clustering is ongoing at pres- 
ent_ but is somewhat ter>tativei It is appropriate, however, to go into 
more detail oh the more conservative conceptual approach of generating 
hbmbgehebus higher-order viriabl^ 

7.442 Hbmbgehebus Higher-brder Var 

The present set bf basic item-type variables; frbm the six topic- 
specific classroom process instruments reveal a number of eldsters 
based, bh harrbwly cbhceived, Ibw inference similarities which are suit- 
able for generating the next higher level s bf aggregated variables, i .e,^ 
thbse which were earlier called "hbmbgeriebus higher-brder variables." 
Three clusters of this type are discussed here. 

Represeritatibri Type ,: A major issue in educatibri_ for some tirtie 
(especially since: elucidated by BrurierJ has beeri that of which type of 
represeritatibri of subject matter is appropriate arid differeritiaily helpful 
in which type of iristructibrial situatibri. Brurier used a typology iri- 
voiving three rriajdr types of coriterit represeritatldri: the concrete, the 
icbnic^ and the abstract, this typdiogy has been extensively discussed 
in the literature related to mathematics edlicatidri arid to cogriitive 
■growth," . ■ ' ■ 

A' typdjdgy of instrucyon situations to be related to this typdiogy 
of representatiqn^ltype is more difficult to identify^^ fr the literature. 
However, a broad typology is^ embedded in the Study's classroom pro-: 
cess instruments, Instru^ may be ciassified ijit^ 

categories: concejDt-orien^ed irtstructi^y^ rale- or procedure-brierited 
iristruction, and applicatibns-Toriented instruction. 

A crbssiri^ of these two typolbgies yields riirie sub-categories of 
variables, brie for each represeritatibri, iristructibrial situatibri pair. 
The preserit jristrumerits yield indices for each of these riirie stib- 
categbries overall , i.e^, aerbss (or fDoblirigJ the six topic-specific iri- 
strumerits, arid, iri most cases, ^ ^m^ bf the^six iridividual topics yield 
iridices.fbr the sub-eategbries for those topics separateryo There , is 
thUs a rietwbrk bf variables able tb pbrtray iri great detail represeri-- 
tatibri typenrid its iriteractibri with iristructibrial sittiatibh typ)e both 
within specific tbpics and aerbss tbpics. The pbssibility bf relating 
these variables to student achieyemerit scores withiri specific tbpics arid 
across ; tbpics suggests a ^^ H^^ jxiteritial for exploring the ei^febt of re- 
preseritatibri type ori studerit^achieverrierit, 

it should be rtdted that the crossing or representation and Instruc- 
tidriar situation types td produce riirie sub-categdries that form - a riet- 
vyork of variabies ddes not exhaust either the variables or Indices that 
can t)e related to that crossing and that netwdrk. Sditiethihg like "row 
sums" arid "cdlumri sums" aggregates can be fdrnied td get bveraii 
indices for a given representation type across Instructidrial types> etc. 
In addition, the network of related indices also makes possible the 
development of ygriables related to profiles 

categories . fromj the^ The portrait that can be used to study the 

effects of representation types is thus rich indeed; 



Diversity . The issue addressed above was of the effect of 

specific types or . blends of subject matter repre^^ in various m- 

strUctiohal situations. A related, but separable/ issue is that of the 
effect of diversity In ihstructibh. Is it better, for example, to pick 
one central represerita^^^^ of^ the subject matter for large portions of 
the ihstructiori or is it better to expose the student to a variety of 
reprosehtatibhs (e/g. , in teaching fraetidhs, representing fractions 
through sets of concrete objects , through diagrams of shaded regions, 
through numerical ratios^ etc.l? Does the value of such, diversity 
depend on the type of instructidhar situation te.g;^ diversity may be 
facNftating in cdhcept deyelopmerit but inhibiting in learning compu- 
tatidnal techniques where se ajgdrithm for extensive drill may 

be the best approach)? Th questidh df diversity (or as Dieries called 
it "mathematical ^arle^ is clearly both ah interesting question arid 
brie of pr^cticaT importance. 

Iri!aices_ of diversity _f^ aspects df instructidn" essentially 

are based on measuring how niany ou^ alterhatives related tb 

that instructiorial aspect were typically used by a teacher in teaching 
the target class . Iri some cases (e.g. , repr^es^ntation* df sutye^^^ matter 
a ^discussed earlier) a typology exists Which partitions or classifies the 
jTierribers of that set bfi alterriatiyes into a set of categories or types. 
In tl^at ease,^ diversity iridices could either be^based on the number of 
types utilized or bri the actual riumber of alteroatjyes utiiized; 
•■ . . .... \ t> ■ , 

To the^ extent that diversity iri iristructiori is fecnitaj[ve o^r irihibi- 
tive> the simple, a-typbl6gic^l iridex should be most powerful. if, 
hdwev^r, the typdipgy captures some aspett of the instruction relevarit 
td the effect of diversity, t^^^ typblogically based iridex may be more 
pdwerful. Fdr example, iristrUctibri vvhich Uses three representatioris of 
a concfept passing from cdhcrete to abstract tb iebriic may relate much 
mdre significantly to achievemeri dbes iristructiori that simply usei 

three rather than one subject , matter represeritatibri but all of the same 
^type, e.g. , abstract. 

It is not possible a priori to specify whether the typqlbgical br the 
aTtypojogicai indices will be most pdwerful iri aby of the aspects .of 
mstruction which yield^ d^ indices and which a! sb have available 

some potentiaily rejeyant ty^ the apprdach here is tb gerierate 

tw^o separate typ^s of diversity index in each situatidhj (1) variety , 
which is always a t^^oiqgjcajjy based index, and (2) focus, which is 
always a simple, a-typol6gical index. 

. The preserit arialysis^of yarlables has iecJ . to seven rjdwest-level 
variety iridices, three related to representations used (in isach df the 
three iristrrUctjbrial situatiori types).^rid four to bther aspects of instruc- 
tibri such as riumber of^ different factors indicated as utilized in making 
specified instructioTi choices (l.e.,^ diversity in method selection dr 
chbice approach) . There^^re, of j:ourse, seven, lowest-level focus in- 
dices cbrresporidirig to these seven variety indices^ but there are also 
three other focus iridices that have Beeri itJejitlfled inisituatlons for 
shich rib relevarit typbl^^ available, bringing the total to ten 

IdwestHevel focUs iridices. 
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These indiees have been called "lowest level" above because it is 
again possible to aggregate clusters of them into higher-ievei indic^^^ 
although, in this ease; sucii a cluster would Ihvdlye mdre heterogene- 
ity and. would have to be either statistic—or model-based. An exanripje 
of such a cluster would be cbmbihirig focus indices for such things as 
diversity in faqtbrs cbhsidered in instructional decisions, in subject- 
matter sub-tb[3ics covert, in prbgram emphases, etc 

diversity index related, to the whole of a teacher's appro^a^ch to^ irtstruc- 

tibh in a given tbpic. Actual ^pecificatfon of such clust^ers must wait 

either for data or further model develdpmenU Fo^^^^^ 

say that the array of seventeen types of diversity indices 

able for specific topics as well as across top^ics) offers the possibility - 

for a rich portrait of the pjace , of diyers|ty in^^j^^^ 

opportunity to explore some quite detailed hypotheses about diversity 

effects, 

bifferehtlitlbn . A central issue in edueatibri generally, as well as 
rnathematlcs education specifically , has . been that of^ inyi^idualizatibri or 
the adaptatbn of Ihstructiohal approach tb adequately respond to indl^^ 
vidua! differehces In the students taught. Issues relating tb ihdividual- 
ij:ation have Included various aspects of tailoring ihstructibri; grbupihg 
practices, assigning different tasks tb various ability groups. While in- 
formation about attifades and practices related to the first two of the 
preceding aspects is sbtight thrbugh a number of itehis on the general 
teacher elasspbbm prbcess questibhhaire> the last area^ i.e., task dif- 
ferehtiatibh, is explored in more detail on the six topic-specific ques- 
tionnaires.. 

As was stated earl ier^ the items of the six topic-specific Instru- 
rriehts seek specific, detailed pieced of irifdrmatibri that r^^^ 
"descriptive mosaic" cdverihg a .variety of aspects related to .a j/ariety 
of specific ihstructidhal situations and instructionai decisjon-making. . 
Much bf this mbsaic relates td the subject matter (conceiDts, procedures, 
appjicatibhs) > representatidhs, techhiques and instructionaj strategies 
typically us^d in presenting various topics; Almost all of these a^^^^^ 
of ihstructidh are pdtehtial candidates fqr djfferehtiatjon decisions In 
order td respond to beidw average students, above average students or 

both. . ; . . ' 

In a iarge proportion of the; items providing the ihfbrmatibn just 
described, additional jnfoj^matjon |s sought : about differeritiatioh related 
to the thrust of each Item; mcluding the- group (above average, belbw . 
average, or both) towards vvhich differehtlatibri Is directed and , bfieri 
some measure of the frequency of, or emphasis bh, sbme particular dif- 
ferencei : : • 

This, aspect of items makes possible dif^rehtiatibri indices ter a 
[arge number of Items, and it also makes^ possible mbre aggreg^^ 
ferentlatioh Indices related tb various aspects of ihstructidh (e.g.^ typ^: 
of representation used, subject-matter.: sub-tppics included both 
for specific topics arid across tbpics. These^ indices are availably ih twd ■ 
forms: - (j) differeritiatibri indices^ which use simple^ unweighted aggre- 
gatioris of presence or absence bf differehtiatibri^ arid (2) ihtehsjty in- 
dices, whiefi are weighted aggregates bf sirriple differehtiatidh ihdica- 

o 
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tors; with weightings determined by frequency and emphasis informa- 
tibhi 



Gdrrehtly, in addition to item^jeyeJand^ Indices^ 
there are seven higher-brder differentiatjoh Indlces^and six intensity 
indices (available across topics and for many speciiRc topics) related to 
most of tjie same aspects^of instruction as werji the djyerstjy in 
Th|s makes possible a detaTfed portrait of teacher differentiation prac- ; 
tices to be related to the ihfQrmaJ:iqn gathered (through the g^^^ 
qUestionhafreJ ^bqUt the^ bther individualizatjoh and to be 

relatecJ, to student achleVemeht arid attitude. This also makes posiibie^ 
as was ; discussed briefly earlier, the ebmbihatioh of differentiation 

: Informatibh with diversity. ihformatibh and other types to provide more 
global characterizations of a teacher's practice through mbdel-based, 

. heterogenedus aggregatibris of variables. The characterizatlbhs of^ 
teacher practice that would emergeVwith varymg levels of aggregation— 
varying topics, relating to yarioUs \aspects of fnstructibh, would seem 
potentaliy to be both descriptively arid explanatorily rich, 

7.4.3 , An Examjjje \ 

Perhaps additionai insight can be p(-pvided through illustrations of 
how some typ[cal Items will be used. Sli^owri on the next pages ^re 
some items from Jhe Common and Decimal Irractioris duestidririaire. The 
fate of these items in terms of the variabl^ just described will be ex- 
■ amined. ^ 

Essentially these items involve on securing key infor- 

mation surrounded by a cluster of diher i terns _\eeki^^^ additidrial infor- 
matibri. The central item examines teacher useYof an array of eleven 
"interpretations" (representations) of fractions^ NThis array dr set df 
alternatives can be classified according to representa^tion typ^^^ a 
, is abstract^ c is icbriic> ^ is concrete)^ Although ste^ in 
this item is riot well balanced in terms of representation type; the 
balapce is much better when this item is aggregated \ith others of th 
same ItSrri tyjDe from the same questibhriaire. (There are ^currently two 
other such items.) The central item -wi]l then contrjbute to indices 
related to represehtatibh type fbr cbhceptual ihstructlori;for the frac- 
tidris topic (arid through it to aerQss-tbpic indices). The additional 
irifbrmatibri sbUght bri freqUericy ahd/br emphasis ehhahcesvt descrip- 
tidri arid rtiTght be used -for weighting the aggregate indices toward 
greater, power. V 

_ Since the ceritral item secures iriforrriatibri relative to a %et of al- 
ternatives dri some iristrUctidrial aspect {coricept refDresehtat\ohs) , it 
also wiii make a cdntributidri td varidUs diversity Jridices. Siricdva typb- 
iogy (of representatidhs) is available, the item may contribute *bth t 
varfety indices and fdcus indices. Specifically^ it wdUld aggregate with 
bthier items of the same type arid tdpic td_ prbvLde ari iridex fon the 
Cy-F (Concept Variety, Fractidns) and CF-F (Cbricept Focus, Frac- 
tions) var[abjes arid, through thehli would cbritribUte tb other hidher- 
brder variables. , ^ 

The; cluster of re[atjd items seeks info rmat ion about how this iri- 
stahee of coricept representation is- the target class teacher Vo 

enhance the achievement of above average and/dr beldw average stUr 
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COMMON FR ACT^ \ ' 

1. Vanoiis ihierpretaiiphs of fraciiphR are depicted oh the left below. For each ihterpreta- 
' tipri (i^^^ pj^ce a check in the box by the response that best describes yoor use of that 
interpretation. 



a. Fraciiont as qiiotienU: 



^ meant "3 divided by 4* 



b. Fractional parts of a cbIiecti6gU— 




I do 
not use 



I use .... *- 
occasionally 



I use 

frequently 



I do 
not use 



I use - 
occasionally 



I use 

frequently 



c^ Fractions as regions: 



d; Ffactions as ratios: 




I do 
not use 



I use 



occasionally 



I use _ 
frequently 



ido 
not use 



I use 
occasionally 



I use 

frequently 



e. . FracHbns as segments: 



f. Fractions as operators: 



K] pp«r«tor rSI 



I use 

frequently 



I do 
h o t u s e 



i use _ _ L 
occrt si on al ly 



I use 

frequently 



I do 
not use 



I use _ , 
occasion ally 



g. Fractions', as repeated addition 
of the iiiiitl 



3. =1+1 + 1 

4 : 4.^4^4 



h. Fractions as decimals:. 



f = ,75 



I use L 

frequently 



TSo b 
hot iise 



I'use 
occasionally 



I use E 
frequently 



Tdo ' L 
not use 



I use 
occasionally 



Fractions as pointis on the number* line: 



i. Fractions as measurements: 



i i i 



I. I I 



H — 



this container holds ^ jt 



I dp 

not use > 



I iise L 
occasionally 



t use 

frequently 



k. . Fractions as number pairs: 



this 



box \ eighs l^^g ' 




three fourths as (3.4) 



this stick is m 



I do 

hbt^iM- 



I use _ __ L_ 
bcca st oh all y - 



l use 

frequent l y 



I do 

not 



I use 



L-:^-jr I us© 
y ~ ^ frequently 
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dents. This additjonal information thas makes poss[bje a contribu to 
a diffejceatjatioci index^ RD-F (Concept Representat[on p[fferentiat[qri , 
Fractions) , for -the topic and through it a contributjon to the across-* 
topics version of the Jndex and to other, hjgher-order variables^ Since 
freqtiehey ahd/br emphasjs in^fo^rmation is also available, this additianai 
information makes possible contribations to intensit) ^ (weighted differen-* 
tiatibh) indices as vvell. ' ' * . 

Still other items in the cluster of related items secure ihfdrrnation 
ph the reasons for use and hori-use of the interpretations involved. 
Utilizatibri of this ihfprrnatioh relies oh a facet model for the item type 
dealing with reasons for ^se/hbn-use. She version of that model is 
presehted in the next pages,_ It Uses fpUr major categories of reasons 
with various sub-categories for each. Within each category two ver-* 
sidhs of reasons are available> positively stated reasons for use and 
negatively stated cduhterpart reasons for ribn-use. The sanifDie state-^ 
ments of each category arid versibri are exactly the choices for items of 
this type throughout the varibus questibririaires (there are currently 
eleven iOther sUch itemsK , 

IThe particuJar items from the example can thus be scored, iri terms 
of the categories (or even sub-categories should that level; of arialysis 
prove fruitful) of trte model. This makes possible a cdritributiori to a 
profile of the factors entering into i^^ choices for this teach- 

er. The model, in fact, provides a typology for this aspect of instruct 
tjon and makes it possible for these items to contribute to diversity 
indices (both a variety^ Ind^^ and a focus !ndex> COF) fdr this 

aspect and, through them, to other, higher-order indices. 

The fate of this example is typicah Fjrst, the central item and 
cluster bf related Items provide item level yar[abjes and add to a firstr 
order descriptive pbrtrait of elements of ihstructiqnal practice; Seconcj, 
they make contributions to a number of different a^gre^ate Imdices (stjil 
of the mbre hbmbgeriebus type bf cluster). Third, the aggregate in-* 
dices til at emerge can be further aggregated both across topics and/or 
to stiM higher-order statistical- br model-based clusters. Next, theje 
indices at ^varying levels^ of aggregation can cbhtribUte to studying 
relatfdriships' within the pbrtrait of instruct ibrial practice (i.e. , i with in 
the Classrbbm Processes compbrierit) arid, finally, they may contribute 
to analyses relatirig the three cbmpbrierits bf the study by cbririecting 
these sort of indices td^tbpic-sjDecific achievesmerit stibscores, to influ- 
erices identified iri the Curriculum Arialysis (e.g.. Does riatiorial pblicy 
affect differentatibn? Do riational syllabi arid exjmiriatidris affect ^diver- 
sity?)., etc. * This cdmbiriatidri df simple^ specific items with cbritribu- 
t ions to a complex network df pdrtrayirig variables would seem to be brie 
of the major strengths of the classroom processes compbrierit of this 
Study. 
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REASONS FOR USE 



REASONS FOR NON-USE 



1. Student: Cdgnitiv2 



U Student: Cognitive 



.1 •! Meaning ' ; ' 

(1) Allows student to associate 
meaning to symbois * 

(2) Provides a meaningful con- 
: text_ for / _ 

(3) Concrete J^nterpretations 
^ are meaningful. _ 



1 .1 Meaning 

(13 Not ah appropriate 

interpretation, 
(2) Ndt appropriate 

for the formula 

studied. 



1.2 Uhderstahdihg 

Knowng-'more tha^^ 

method aids understanding.. 



1 .2 Uriderstanjiln^ 

(1) Learning rules 
doe s not a id _ 
understanding. 

(2) premature^jearnjng 
of rules interferes ; 
with uhderstandlng. 



Il3 Structure 

(1J AJIdws student to distin- 
guish .. frbrri 'j' 

(2J Allows studient to see 
similarities^ of to 



(3) Allows student to relate 
to 

[U) Alldvys student td use 
previdusly acqujred 
xkridwledge td learn 





2. Student: Affective 


2. Student: Affective 


(1) Interesting (and/or appeaf- 
ing) to students. 


(1) Though students wduld 
dislike it. 



(2) Easiest for students to 
_ .understand. . ^ 

(3) Easler /foi^ students to 
have definite rules. - 



(2) Might confuse students. 



3. Subject Matter . ' 3. Sub ject Mattery 



3,1 Effectiveness 3.1 Effectiveness ^ 

(1) Felt it would be effective. . (1) , It Is m effect ivfe^i-;; 

(2) Most effective technique. (2) [t Js too time con- 

(3) Powerfu! technique for suming* 
mental: arithmetic *(br did they mean to 



teach rather? thari to 
use; i,f, sb^ then ^.^) 



Cont'd.) 
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^ 3i2 Justificatibh^ 

(1 j Mathematic Justificatibhs 
<> are irripbrtanti 
(2) Provides rflathematical 
. justiflcaiibhs for the 
step's of , 

3.3 Applicatidns 

(1) Application Sttudehts likely 
tp see iri the ^future, ' 

(2J NumerbUs Jhterpretatib 
;^cil|tate appljcatibhs. 

[3J Students need practice 
in describing natural 
phehomena through 
mathemati symbols. 



4. School /Teacher System ^ 


4. School /Teacher System 


4.1 textbook 


4.1 Textbook 


(1) jt'sjn the textbook 


(1 j Not emphasized or 


(2) k^s^ny^hjsjzed or^r^ 


^ recommended in 


mended in the textbook. 


the textbook. 


a;2; Syllabus ' 

(lj it's reqtiired in this schobi 


; 4;2 Syllabus 



4; 3 Teacher 4^3 Teacher _ 

(1J Not familiar with 

the ihterpretatiani 
(2J Did hot think of 
using it. 



•4.4 System Demands 4.4 System Demands 

" - (1} Mb tim^ to present 

. * all % 

* (2) Required ^toack- 

grbUrid students 

. ^ did hbt have. 

I' • ' . ^ . - 

: : : '. : ' - . l 
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8i ComjDorieht III: Student Outcomes (The Attained Curriculum) 

As has already been stated arid iilustrated^ the Study probes the 
cUrricuium at three levels. The curriculum analysis seeks to portray 
the curriculum as intended by the. educationai system; the ciassroom 
process compprierit p^ the ways in which' the curriculum is impje- 

merited by the teacher iri the classroom. The third Gdmponerit^^^^ 
subject of this section, expibres the nature and extent of ^urricular 
goals as they are demoristrated_by_student-- attainments^ — 
measures. ^ - 

^ Two broad ciasses of measures are utilized^ cqgn^ 

tive; Proposals analyzing -the data at the item and subscore levels are 

given in Section 14; 

^ ^ f . ■ _ ] ^_ 

It is envisaged that limited supplementary dat^ from students will 
be sought ih the areas p7^"generaj backgro^n^ ijTTfbj^mat^^ 
breadwinner occupation; .etc;); and classroom process (including oppor- 
tunity to learn arid uses of harid calculators). < ^ 

The following aspects of student outcomes are of particular -Iriterest 
iri -the Study: beriehmark cbmparisbris; growth duririg the academic 
year; linkage with classroom process; sex differericesi 

8.1 Beriehmark Cbmparisbris 

Iri jTiariy lEA cbUritries^ a major cbricerri is a comparisori of the 
status of mathematics edUcatibri riow that of ~f if tee ri years ago ^ the 
date of the first survey. Although many chariges have taken place in 
mathematics edUcatibri ih the past decade and a half, there are little 
empirical data tb dpcumerit the extent and magnitude of these changes 
_^ri irite rnationai perspective. . T 

Cognitive Measures , "Anchor items," that is, items used in the 
first survey,;^ will Be ^selected for the purpose of comparing student ' 
achievement then and now.- Criteria for selecting items from tlae first 
survejf^ wiii incjude representation of ceils or curreht^ridj and 
satisfactory psychometric properties; Items wijl .be sought that can be 

classified as follows: ~ 

' bow Level High bevel 

Population A: Arithmetic 

Algebra : 

Geometry 

Measuremerit 

Probability /Statistics 

Pbpu la tibn B ; ; Numher 

A^lgeBra ~ 

Cebrrietry - ' . 

• Arialysis 

Statistics . 

Appendix C provides further informatidn on the anchor items. 
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AffectivB Mea^ res , The change^ of focus for the affective re- 
spbhses has impi ications for how the hew Study will dovetail with the 
bid ohci A harhbyvirig of focus implies the heed to generate hew scales' 
ratKer than simply re-admrnlster the old ones; Still, It is desirable to 
have Jtems cbmmph to bbth surveys to fDrbvide a basis for cbmfDarlsbhs 
between what exists how and what wis' found fifteen years ago. Al- 
though the bid scales have a different focus, there are items in those 
scales that can be utilized in the hew survey. A goal in the ebhstruc- 
tidri of affective scales is to include thbse items that haye fuhctibhed 
well in thje past but to supplem^ them with items and scales that more 
. closely fit the goals of the new survey, " 

8 • 2 Growth During Academic Year \ 

A . growth \measuhe for classes during the academic year is plarihed 
jnterhatidhaHy for Pdpulatibh A and as ah ihterhatiohal bptijbh fo^^Pbpu- 
iation B. The primary justificatip grbwth,measUre_ resides in 

concern of the Study for the classrbdhl. In order to focus upon the 
classroom, what happens there and what studerits learh> it is essential 
that data which references the ^c^^^ as attaihed by the student 

entering the class be available when end of year measCires are obtairied. 
Hence, a pre-test achievement and attitude measure at the classrobm 
level w|n be sought. Some have proposed using such measures as an 
index of school "effects": emphasis on education; qualifications of 
teaching staff; impact of supplementary resources^ The primary 

intent,; however, is to ^se pre-tejit measures on classrooms as a covari- 
ate in [sblatjng learnings which have occurred In the classroom during 
the school year. 

8.3 Sex Differences [as related tb issue bf betweeh-stuHent analysesj- 

"Wdmeh and (or in J mathematics" is.a tbpic receiving cbhsiderable 
attehtibh in several cbuhtries at this time, accbrdihg tb fDrelimlhary 
Jdfdrmatibfc re^^^ by the Curriculum Analysis Grbup. While this 

topic is only illustrative, it is "Jmpbrtaht that some bitween-student 
analyses be possible ev^n though the main thrust of the Study is at the 
classrddm level. 
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9. Versions of the Stady 

As^ work o^n t^he Stady proceeded, certajn modificat^ in the 

Study have emerged. Feedback from national com^mittees has pointed to 
the need for f^^^^ thf selection of items^ fo 

test, and prqbab[y for a more complex structare^f the instrament for 
the bngitadinal stady Jsee Section 12K The demanjJs whjch the pro- 
posed study will likely make upon nationaLcenters anjj schools, in terms 
of administrative details, in terms of teacher, time for respohdihg to the 
classroom process questlonnarles thrbughb school year, arid [ri 

terms of mairitairiirig close contacts between schools and national ceriters 
all point to two needs: (1) a "feasibility run" of the full Study iri at 
least brie cbUritry, arid preferably several, should be cbriducted tb 
dembristrate the mariageability bf what is prbpibsed [2} every cburitry 
iriteridirig tb participate Lri the full study shbuld pJari .tb uridertake a 
full scale dry ruri bf all iristrumerits_arid data cbllectibri prbcedUres. 
This shbuld take place lri 1980-1981, Firially, problems iri furidirig have 
continued to irtlpbse further delays bri the time schedule. 

The Jriterriatibrial Mathematics Cb^^^ therefore agreed > at its 
jahuary 1979 meeting (see Section 2.3.2) to view the Study as having 
two Versions, or elements. One is.a rej^Juced, or crbss-sectibnal study> - 
which makes fewer demarids Upb in. terms df data cdjle 

tion, testing time (testing takes place "^njy once during the school year) 
and costs (administrative, printing, etc, j . This version of the study 
consists, essentiajiy, of Components i_ Cthe Aurricul^ analysis and 11 1 
(the student outcomes survey) and eliminates the investigation of in- ^ 
siructionaj practices (the classroom processes component j for Popula- 
tion A* ' ' \ " 

For Pbpulatibri B the versibri bf the Study remairis for all eburitries 
as aririburiced in Bulletin 3, The implicatlbris of these plans for tlme- 
tabllrig can be summarized as fbllbws: 

ii Pbpulatibri A [erbssrrseetibrial) : proceed bri timetable aririburiced 
lri Bulietiri 3 arid reproduced iri Sectibri lE^ bf^ this Bulletirii 
ii. Pbpulatibri A (IbrigitUdirialj : cbridUct Tull scale dry rUri dUririg 
1980-1981 and cbllectibri data brie year later than iridicated iri 
the timetable. _ o 

iji. Pbpulatibri B: Since a IbrigitUdirial study is a riatibrial bptibri 
briiy i, cbUritries dbiri^ the crbss-sectibrial study will proceed 

./ bri the schedule iridicated in the timetable. CbUhtries inter- 
ested, in the johgitudihaj study may wish to get in touch wijh 
the U.S. Natiphal Cbbrdinatihg Center, University of illihdis, 
Urbana (attention: Kenneth Traversj fdr .cdp of the cjass^ 
room process instruments being developed for the calculus 
and other topics at that ieveij. Recommeaded dates fdr 
longitudlnai study, population B, ^re 1980-1981 . 

9.1 Populations Xr^ss^^ectlonai^ Version 

Some countries have mdicated that they wish to take part in only a 
cross-section a| sjudy at the Popjjiat[on A leyej. At' least two of these 
countries have Indicated that they must collect data in the first half of 
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1980. 71^^^ jhterhatidnai Mathematics .Committee therefdpe made efforts to 
have instruments ready December 1979. 

Cognitive Testsi- knowledge of Mathematics 

For each populatfon, final instruments were devised to address the 
needs of the cross-sectionaLstudy. 

Pop u l a t i on A Te st 

This test consists of a core form and four rotated forms (A, B, C, 
D). The core test^is dev[sed^^^t^ P*"9y'de a relativeily comprehensive 
measure of mathematics achievement with the balance of items frbrfl the 
V and I ceUs of the [hterhatibhal grld^ assigned by content and behavior 
stratification to the rotated formsi The number of items in the 5 forms 
resfleets the weightings of the V and I cells , in the grid and includes 
about 30 items from the First Study (these are the "anchor items"). 

Population B test ^ o 

As this tesst was brigjriaHy structured (see, for example. Bulletin 
3^ page .29) it cphsisted of, 8 forms of 15 Jtertis each. The seven forms 
were constructed, by raridbiri' assigrim^ of items frbm jhe iriterriatiprial 
pool . with stra tificatibh bh ' cbnteht and behavior. The eighth form 
consisted entirely of items dh the calcujus. 

'Thbse siveh forms, were cbhstructed Using the jnternatibnal grid as 
a blueprint, and for some countries, provide an adequate reflection of 
cUrricUlar emphasis. For dther cpUntr|es> hdweyerv thdse seyeri forms 
do not provide : isufficient emphasis on the- calculus. Therefore, for 
"those, countries all eight forms v^ere to be used. 

Two concernis caused th e IMC to progofe a new structure for test. 

First^^he use of a test forrn defined by con^tent^ (as opposed.Jq random 

ass^lgnmentj iQtrod^ces methodojogica[ problems. ^ P^r _^X3?Pj§'_ 

. sampling ej^rors of the^ estimates of item difficulty on a form where there 
Is no random assignment will have inflated J tern in te red r 
Thus; a substantial amount of Imprecision, and perhaps bias, would be 
Introduced into the estimates and subscbre parameters. 

Ah additibhal .ebhcerh had tb db with administrative practicalities. 
The calculus form is, bf course, quite differe'ht frbrri the bther form arid 
cbUld caUse cbrifUsLori or coricerri .in^ the classrbbrri. It is alsb likely 
that the^calcUIUs form wbUld ..take. Ibriger tb answer than the other 
fdrms. ' 

The PbpUlatibh ; B test has therefdre been restructured by the IMC. 
arid preserited tb the cburitries for ; cbrisideratibri, TJnis restruetur'ed 
versidri is prdpdsed fdr use by all cdUhtries participatirig. in the crdss- 
— sectibriaUstudy-.™ilT-hbse^^^ 

the IbrigitUdinal study, will have available , either the versibh bf the 
PdpUlatibri B; cbgriitive te$t preserited iri Bulletiri 3 or sbitte bther struc- 
ture, which responds to their Interests and needs. For example, a core 
test which yields sUbscbres bri such tbpics as trigbribhletry, systems bf 
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equations arid elementary functions may be appropriate. The IMC will 
assist in thei exchange of information between countries cohcerhihg 
plaris for the Population B longitudinal study. 

The new international version of the test consists of eight forms of 
17 items "liach. Each form is a stratified ;(By content and behayidr). 
raridbm sampl^ of the item pool which was devised on the basis of the 
international grid. ^ . 

Attitude Scales 

Four scales in their recommended final form are as follows: 

Mathematics ir[ School ' " ''' i 

Mathematics and Socjety 
Mathematics' as a Process 
Mathematjcs and Mylself 

Computers and Mathematics , : c 

Details on these scales, together with results of the pilot test'mg of the 
items^ are available in the IMC document. Report of the Attit ude Tri al s , 
December, 1979. 

Questionnaires 

During 1979, National ciriters had two opportunities to comment on the 
school, teacher and student quest ion rial res cross-sectional 
versibh. These instruments were finalized In September, 1979. 

DppbrtUhity-tb-learh ratings 

it is plahhed tb gathei^ these ratings from the students and 'their teach- 
ers.. • • ' 

Sampling 

A sampling mahual for the c/oss-sectibhal v^ been sent to each _ 

country including^ a timetabje for the riegotiatibnsv with the samplirig 
referee. This timetable recommends a bhe-year lead time be^^^ data 
cdllection^tb allow for drawing the samples, securing permissidri from 
schools, and making other arrarigemehts. 

Admini str a tive m a nuais „ 

Drafts of these manuals were sent to National Centers in early 1979. 
f=inallzlhg took place dUrihg the September 1979 meetings. 

Data P r ocessing 

Despite the cbntinuihg lack of fuhdihgi> cdntingehcy plans have been 
driwn ap by the Irit&rhatibhal Cbbrdiriatdr. Dr.: Larry Nelson, Univer- 
sity" of OtagoT New" Zea!and7-h as~been-contracted.. to-p r-ovidej:ig^uida.Qce_ _ 
for the data mahagement. One of the Research Officer positions, within , 
the Gbordihatirig Unit has been redesignated to_aiiow for the appbintT .. . 
riieht bf ah experienced programmer. Budgetary, provision has been 

■ , ■ ■■■':.„.. ,', . ■ ' 
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made for the necessary, cbmpater'^ time and Negotiations are cbhtihdihg 
wjth the Department, of ^Education^^^^^ the instajlation^ of a visual display 
un[t ^nd liar^d copy term of Edjicat|bh; Diffitcil- 

ties are bejng experienced^ in finding a sodrce of fandihg to parchase 
Osiris IV as an editing and fjie building package. Once this is solved; 
work will begin oh establishing the necessary routines with dummy data 
whjch are already prepared. Efforts will be made to cbbrdihate; the 
work of this unit with that of the Data Prqeesslhg Gehtre for the lohgi- 
tudinal study at the University of Illinois 

9.2 Longitudinal Version v 

'_ _ Develbpmerital wbHk bh the various iristrumehts will proceed in 
1980. / : ; 

a} BackgrbUrid questionnaires (School / Teacher / Student) arid attitude 

■ scales- / _ ' ' _ ■ . ■ \ ... ■' ■ " : - •- •*■ ■ 

. The qUestiohhaires as jDrbdUced for the crbss-^sectibhal study will 
fo rtt) a substantial basis for the ful 
tb) Classroom prbcess^q . 
These will be pilot tested in about six countries with an attempt . to 
sample diverse instructionai practice. As a result, cdnsiderable 
"ihterriatidhalizing" of the instruments In order to account for dif- 
ferences in instructidri 5hdu!d emerge, 
cj ^ Cbgnitlye Instruments . .\ v ^ 

Suggestions for further development are .outline in Section 12. 
The IMC wiji strive ^^^^^^^ instrumentation in time for dry run 

beginning in September 1980. ^ 

For countries which, of necessity data coi lec- 
tion for Population A during 1980, interim forms of the instruments 
(re^luced studyj wiJI J^e^^^^ December 31, 1979. jt js expected 

that the overlap, in term^ of^ items, between these instruments iand the 
final . straments will be jfiaximal. .That Is^ ^a 90% of the items will 

be cbmmbh tb both sets of Ihstrumerits. 
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.. . j0i Research Hypotheses . . 

1 6; 1 i.eveLl^. Educatibrial System - : 

The data frbiTi both the Cbhtexts and Status SUrv^yf. of the 
riculUrri J\halysis will make pbssible the exalnihatibn in a descriptive 
sense bf , relationships iilustrated by such research . hypothesesr as:" 

1 . It is expected that the existence .bf riatibhal examihatibhs is 
accdriiplished by mfhimai variatidh in cUrricuiar jDatterhs and character- 
istics, ; . 

2. It is expected that currlcuiar change takes, place more readily 
where such change is supported, and to a considerable extent directed, 
by external examinations. 

3. It is expected th^t more variation in curricula exists wher^ 
there Is ji number of dlscr^^ with different foci and 
constituencies as contrasted with ju^ there is one 
examining authority or no system-based examining authority, . 

•10,2 Relationships between bevel I and bevel II 

. 1, ^, It . is expected that the more overt and explicit the control in 
theVeducatlbhal system; as Indicated by the presence of a Hatibhal cur- 
riculum; a c^htralized inspectorate and a system bf external examina- 
tibhs and/or grading systems, the less variation will Be: f^urid between 
teacher bpihioris bh ctirrietilar Issues, jtistifieatibri . for and use of ih-^ 
strtictibhal strategies ahd both within and between classrbbrri Use bf 
ihstructibhal strategies and resbtirces; * 

. 2i It is expected that mbre limit^ resburces> ih: terms, bf ihsti^ 
tutibiis and facilities for, say, teacher ^dUcatidh> will lead to iHpre 
cbmmbri sbci^lizatidn of teachers ^ hence less variation in teacher ^atti- . V 
tudes, opmidns, expectations instructidnai behavidrs. 

One aspect 'of the linkage, between the curricuium at the system or 
natlona! levei arid the classroom is that of teacher coverage or "oppbr- 
tuhlty-td-l_earnl^ 0 measure is to regard it as a 

index , of' the curricular validity of^ the international tests (th 
th4 extent to which ^ subject matter has actuaijy been a 
country's classrooms; the more appropriate is^ that test as a measure of 
student achfjeve^neht In that cbuntrycarid as a cbrbllary, the more mean-: 
ingful are crbss-nationaJ cbmpar JNA t^st)i As In past lEA 

studies, we would expect that teacher covisrage .will be a relatively : 
ppvy'erful variable In terms of aecburitirig for variance in student a- 
chievementi 

There is ahbtHer dirfectibn in which : ihterfjretatibh of the teacher 
coverage index could move. One ebuld speak of cbhgruerice. 'between 
the intended and Jmplemented curriculum as^ s^ sbrt of measure bf 
tea cher^^bmpe tehee - or effectiveness,, and . cfntemplate^ !:Sbme sdrt bf 

"ranking of~h"atlb'har~^systems-bf"e^ 

which the systems' teachers Implement the hatibrial curri ' - ^ 
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Such an mterpret^^^ ^Itfa Us, however. A fun- 

damental jDroblem is tha is not ^a mechanistic process whose 

success depends upon the degree to which desire^^ Information can be 
delivered^ or "laid on" students. The ess^entia[ dynamics of a class- 
; rpom ^ thg sensitive iriteractibh between teacher and student/ the judi- 
qibus selection of appropriate subject matter and IhstnjctionaL agprdach 
for a given da^ are among the factors which are to be taken into ac- 
count Jh an edu Teacher coverage, is an important 
me_as_ure> but is. only one of a vast complex 5f cr^itical variables which 
enter into the mix for effective teaching and learning, 

ib.3 liypotheses Linking System arid Student Outcomes^ 

This cjass of cdmparisdris investigates possible relationships be- 
tween characteristics of the curriculum arid accbmpariyirig studerit butr 
comes ; 

: Attitudinar Dimensions - 

there [s cbrisjderable overlap between the attitude scales of the 
first and second mathematics study. ThLs should . permit useful bench- 
mark comparisons for countries which participated In the. first study. 
Iri particular, countries^ in which student attitudes were mariifestly 
/riegatiye iriJhe first study may wish to identify possible shifts bri these 
measure^^. • The Mathematics as a process scale, designed to measure 
orieVs view of the nature of mathematics may ajso be examined iri the 
light bf curricular emphasis in the various, countries. Orie might hy- 
pothesize, for ej<ample, that curricular emphases [n Alpha and Gamma, 
combiried with the above projected profHes of classroom -process in those 
cbUritries, would be accompanied by Mathematics -as a Process outcomes 
such as suggested belbw. A / : 

. ^ - . . Mathematics as a Process \^ 

This scale is irit^rided tb bbtairi a: measure of the stuA 
denVA perceptibris abbUt the hatUre of rriathernatics. If is\ 
hypothesized to reflect differences iri cUrrictilums arid in \ 
era ssr bom pr presses . For example, emphases bri mathematics as 
a creative eriterprjse^ arid ^^ d approaches that ericoiir- i 

age tjpe^n-gnded irivestigations on the part bf. stUderits, are 
expected to impart a view of mathematlGs that is c5yriamie arid 
creative. / 



High 



Mathematics 
as a 
Process 



Low 



ALPHA 



BETA 



GAMMA 
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- Cognitive Dinierislbris . : ' 

the mbst popalarized asfjecti of tViis sort of study are, ahaoubt- 
edly, the student butcbrrie measures, and particularly thf cognitive 
aspebts. 0 in view of the ebnsiUerable cu rricul a r activity over the past. 
20 years, bhe shbuld expect attention to focus upon comparisons be- 
tween student achievement "then and now." The results of Jhe National 
tongltudlhal Study bf Mathematical Abilities, which examined student 
bbtcomes over a five-year period beginning m 1962 for curricuiums 
based on "cbhvehtibha!" and "modern" mathematjcs textbooks^ suggest 
the sort of findings bhe might eXfJect for the United States^,, and, 
assuming reasonably comparable defimtions of "eoiiventionaj ^^nd 
"modern'!, and accompanying implerhentation in the classrbbms tbold 
assumptions, to be sure!) one might ;expect that countries which have 
uhdergbrie extensive ••modernjzing" of the curriculum will exhibit stu- 
dent butcome measures which have increased on the upper levels .of 
ebqhitive behavior- while at the same time some loss bf ground on the 
Ibvver levels might be antici^pated. 0n the other hand, countries which 
have experienced less "modernizing" of the cUrricUlUm might expect less 
shifting across behavioral levels and (agaih^ bther factors being equaL 
which they never are)- more congruence of student outcomes between 
the first and second studies. 

The figures on the following page present mean achievement scores 
for students in "rribderri" iTextbook T.) and "traditjonal" (Textbook 
T ) textbbok groups, trans formed to standardized scores having a mean 
o? 5b and standard deviation bf ID. The circles in each^figure repre- 
sent textbook groups. Therefore, the. figures present profiles illustratT 
inq the relative position of the, varibUs groups on 37 cognitive scales 
used In the survey. In Figure 3 the darkened dot represents a 
"modern" text, the SMSG program. Nbtice the relative positlon of these 
groups bh the lower cbghitive level as to the h[gher levels. In the 
secbnd figure (FigUre 4) the. darkened dot ^represents a conventional 
text. The relative position of this group is in many respects a striking 
contrast tb that of the modern text. . 

(The follbwihg two figures are frbm E.JS^ Begle and J.:W. Wilsbh, "Evalu- 
ation bf mathematics programs," - MatbematlcsLEducation , Sixty-ninth 
Yearbbpk of the Nationa] Society ^for tne btuay ot taucation, 1970, 
pages 396 and 398, respectively.) 
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Figtire 3 Profile for Textbook Group T 
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(X-Populatron; Grades A, 5, 6; Years 1, 2, 3) 




Figure 4 Prdfile for Textbook Tg 
(X-Populatiori; Grades 4, S, 6; Years 1, 2, 3) 
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item bevel Infdrmatibri A 

M has already been stated in thi^ bulletin ISectjoh ^.3)., and 
elsewh'ere (see, for example; Balletin 3, pages 36-37), the International 
Mathematics Committee v?lshes_to focus upon simple description of but- 
com§s at -the subscbre or [tern level In an attempt to present findings 
which are most easily understood and communicated. There ^is also the 
intention/ however, to preserve the data in a ft)rrnat which enables 
analyses based upon a variety of models. 

One of the pqtentiany most valuable sources of in of 
' the f^rst mathematics study, which^still remajhs to be fully mined jnd is 
reported In Appendix II to .the report of that study. Is the item data 
indicating those countries in which each item vvas hardest and m which 
it was easiest. A sample from the Population \l ' (our Population A) 
results is provided. r V 



' i. A box h«j A vblumc of 100 cc. 'Anotferbbx « twice JW 
long, twice is wide miid twice asltigh^ Hjow many .cc is the 
\ voUitne of the iccond box? Ans. 600^ ^ 




H arder in 



'Aistralia, Sweden, 
United States 
Austnjia, Svifedcn, 
United Stato 



\ 



8. bn^the scale lo the nght, 
the reading indicated by 
the arrow' is belweeri 



A. and 52 

B. 57 and 58 
e 60 and 62 



t ,. ni . i„M ,., y ,, | i, i . ni ^ 



D. 62 and 64 
•E. 64 and 66 



la ,42 :45 B. D giigiand, Jap 

lb/ ,44 .44 B, 1> England, Jap^n 



Bcigium^ France 

Bclgiom, France, 
Israel - 



\Drie\ can offer many hypotheses^ _fDr explaining the differential 
difficuity W th^se items in various countries. If It can be assumed that 
Item i has more of a "real World^'^Tjavor * than Item 7, what is it about 
the 'turricillum in England and Japan which might accburit for the rela- 
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tiye. ease of the scale reading task? Is the appMed nature of mathe- 
matics much more ^vidept m the carriculum of those counrries Jthan jn 
Belgium and France? Item 7, oh the other hand^ has its own intriguing 
characteristics. Does success , on this item reflect understandihs of 
vx^lume in any deep seHse or the ability to use forrridlas? And again; 
what In the curriculum might accbunt for the differences? 

Certainly, ihstrUctibnal effects must be taken Into accodnt as well. 
The kinds of experiehces in which children are engaged as lea^nihg. 
takes- pjace-wduld--stjrely^^ on items iucfi as 

these. Children who have actually stacked blocks arid counted, as they 
learned the concept of volume could be expected^ generally, to be more 
successfuj at solving problems like the brie posed m Item 7. But/ 
the other hand^ prior experience in applying mather^atical formulas 
might ovfer-rule and account for the success of, say/ the Belgians. 

\ The fbl lowing hypqt^ been - proposed by the Curriculum 

Analysis Group for the attitude scales. 

Gbricerriirig relationships within measures : , ^ 

1. Strbng pbs|tjve correlations are expected fcHetween: 

a. Jlathematics in School (ease, liking) and Mathematics and 
Self; ^ . _ , ; 

b. Mathematics In School (Importance) and Mathematics and 
Society; ^ 

c. Mathematics in Sehbol (like) and^ Mathematics and Self;" 
_ d. Mathematics as a Process ^rid Computers and Matheirlatics 

2. Tfie strbrigest correlate of- growth In achievement wilK be 
Mathematics arid Self^ 

Cdncernihg within cbTiritry relatioriships: 

a. Characteristic: Hatiorial profiles bri the :a.ttitude score will 
be foUrid. _ \ 

The$e. prof ires will help prbvfde a corite^^t for national 
adhiex^merit" s^resi \ 
c. Corigruence b^weeh teacher arid stud^ pcbflles for the 
. . . commbri measures (Mathematics in School arid Mathematics 

as^ a Process) wNI be foUricJ withiri cotiritrles. 

Concerning within clais^^r^^^^ 

a.*" Characteristic profiles of classroom . I attitudes will be 
identified, and be\fbUrid to Vary according to: 

(1) type of schoolv 

(2) type of program 

(3) teacher characte (a^e> sex^ traihirig, rriethbds 
used in class, attitudes) ^ 

(4) ^ype of curriculum 

Cbricerriirig ^betweenfi^u^^nt relationships (note: attitude 
items are common^fo ajTstddents) . 
\ Individual pattSrris of attitude will vary according tb: 

a. achievement • 

b. sexL / ^ 

c. sbclo-^ecbribmic leyei arid parental education 

d. expected eddcatlbri 

e. ^/expected bceupatibri 

f. birth order arid family size^ ; 



ERIC 



50 " 



The ratidnaje for these hypotheses comes from a^^^s^^ 
attitudes Awil I result from curricuium and Instruction bat that individual 
variation from the group will result from individual characteristics. 
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11i ^; Sampling issues 

-...^-^^■J!lg^.J^gSffl^ ( i ng;=^d es ig n--a h d-- 

guidelines for irriplemehting the plan in each country are to be found in 
the sampMng manuals. Persons requirma, information on procedures to 
be followed in drawing '3iamples of their own countries, or on dther 
aspects of samplihg should refer to those documents. The manual for 
the cross-sectibhal ^version of the study was prepared by the jnterna- 
tional Samplihg Committee and circulated to all . countries for initial 
commentary iri March 1979. .The :final version was circulated in June 
1979. The Internatibhal Mathematics Gbmmittee is |ndebtfed to the Sam- 
pling Committee for the vvbrk represented by the sampling manual . 

The present sectidh summarizes certain of the main features of the 
sampjln^ design M ehu in ddeumehis from the Australian Council 

of Educational Research (notably, ^M. J. Rosier, K. N. Ross and J. P. 
Keeves, "The Sampling Design," 1978 in Sections 11.1-11.9). Coinments 
on the design, particularly as they relate to the full : (longitudinal) 
study, are drawn frt^m a paper by R. Wolfe (1979), SiBctions 11.8- ' 
11.13. [t !s intended that the issues raised here help highlight the 
particular problems' of sampling which attend the aims and design of the 
ibhgitudihal study. 

11.1 Purposes of the Study ' r 

The purposes of the Study are outlined in Sectibri A above. As 
indicated there, data to address these purposes will be obtained 
through questrohhaires and written documents "at the natibrial br system 
level, from teachers and coordinators of mathematics within Schools and 
through _tests, attitudes scales and questionnaires from st udents. 



11.2 Procedures 

The statistics used tb summarize the data cojiected by the Study 
will, in the mairti be prbpbrtibhs responded to in a specified way to 
items in tests^ attitudes scales or questionnaires, or wi[j be ^^^^^^ values 
^5^^ Analysis^ of the classroom process 

da ta ^ w i n Jnvol^^^ 

the main report of: the Study in seeking to account for the effects of 
currjculums and instructioh on learning. The techpical volume wiJi 
exhjbit a- variety of models used^tb explain the classroom process and 
student output data. 

_ __^There will be an emphasis on the descriptive and comparative as- 
pects bfthe_ data c^^^^^^^ profiles for cognitive 
test items, attitude scale items, student and teacher views and class- 
room and school practices. It is intended to present these findings as 
simply as possible so that the reporting will be meahirigful tb an atadi- 
ehce of mathematics educators, curriculum cd-brdihatbrs and teachers; 
However, there wiji also be an emphasis on the explahatibh bf change ih 
student performance over a schgoLyMr^ T;^hu require the^deyelop- 
ment of ^ statistical mode|s: to account for change, W wi^l I ai th^ develop- 
merit of appropriate' analytical procedures for the examiriatibri of the 
models. ^ 
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To the extent that this study seeks tb make ebmparisbhs with res- 
pect to performance in mathematics within a country between 195^ and 
1 Q « n o r r^r^p^ r j^gmj^hft t wi^Pja^lffet^en^ 

sary to maintajn cqmparabilUy in definition of target pbpulatipris and in 
measures of student performance both between countries arid between, 
occasions. . ' " ■ ■ 

These many constraints on the design bf the study^ arid^ the sampi- 
ing plan. In particular, will be difficult tb satisfy, Cbrisequeritly , it 
will be necessary to allow each of the partjcipatirig cbuhtries sortie 
latitude to select bptibns provided in the design of the study and 
incorporated Jn the sampling plans tb answer the specific research 
q^uestions of greatest relevance to them arid tb cbridUct the study withiri 
the financial prbvisibri available tb them. Some of the options available 
are outlined In subseqaerit sectibris, 

11.3 Sample Sizes 

Previous lEA research has iridicated very^ con^inan^ that jh any 
multivariate analyses seribus difficulties are encountered if the number 
of schools iri a betweeri-school analysis fal is below a level of about 70 
schbbls. Furthermore lEA experience suggests that consistent ah^ 
^ meariingful findings are only obtained when about 100 to^ 2013 cases are 
used iri any analysis in which more than a very limited ri^^^^^ 
predictbr variables are introduced into a regression eqditjbh._ Thi^ 
experience is in agreement with the advice given ^y Kerliriger arid 
Pehazer (1973, p. 46). Moreover , experience indicates that tb have 
less than 15 students from . any school or classroom yields uristable 
results and to include more than 30 students frbm ariy school or class- 
room provides redundant information. Thus ari intact class group of 25 
to 30 students, with th i^4oss of c a ses hptwfif = iri the-pf-e-tes itfi g arid the ^ 
postrtesting not exceeding 10 students to yield a fjriar group size of 1 5 
to 20 students, would appear a sburid basiivbri which to collect data. 

Nevertheless, it is important tb recbgriize that differerit National 
Centers have different levels of fiharicial support available to them. As 

- -B^co nsequen€e^fe^^j#^^^be^^ 

In the design of samples, sb that_ rib ceritre would, be prevented from 
partrcipatibri oh the grburids— that^firiaricial sUppbrt-wpul^^ 
able for testlrrg a large ehbugh i^dy bf studerits fbr ful I analyses to ^^^^^ 
carried but. Where brily a sample of limited size can be tested^ certa|n 
types of arialyses bf the data may riot be pdssibie but other important 
results can clearly be bbtairied. Yet it would also be important to cau- 
tion National Centres : that iri. the design of a sample, the use of too 
large a sample irivblyes riot brily uririecessary expense but cqlje^^^ data 
that are, in the mairi, reduridarit. The sample sjzes disc^usse^^ 
sectibri are those seeri to be the optimal sizes for carrying out ah ef- 
ficient arid Urirestricted irivestigation. Table 1 1.1 records what is seen 
tb be the optimal rarige withiri whjch anj/^ sampjing of schools, class- 
rooms arid students should take place together with a reebmmerided 
sample size. 
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Recommended Sample Sizes 
Table 11.1 - . 





tower 
Bound* 
(Rj 


Lower 
Bound* 




Recommended 


Upper 
Bound 


Recbmmehded 
Total' 


bcnools 


76 


70 


100 


150 


100 


eiassrbbrns 












within schools 


1 


2 


2 


3 


200 


Students 












withiri schools 












Pre^test 


20 


20 


25 


30 


5,000 


Post-test 


15 


20 


2b 


25 


4;000 


Total students 












in analysis 1 


,050 : 


2,030 


<*,000 


11,250 




Total class- 
ropins in ana- 
lysis 


70 


140 


200 







* *Ldwer BbUhd (R): For reduced study. Does riot provide for dis- 

eritariglirig school arid classroom effects. 

..,'■«■..'■ 

Lower Bound (F): . For full study. Enables 'Soiatibri of school arid 

classroom effects. 



In the design of a sample,, a National Center faced with limited re- 
sources should consider carefully which aspects of the arialyses it 
wished to forego arid thus whether it wished to raairitairi ari adequite 
number .of schools for ah effective, be twee ri schools arialysis., ari ade- 
quate number of classrooms 'for a between classrooms arialysis or an 
— adeqtme--Tiamt5'er^ 

tics. If a National Center were_ribt cbricerried with, dlslinguishirig 
^-betweennhe-^fffecTs^d 

schools It wduld restrict its sample to. brily .;brie classroom per schbbl. 
[f, however, a Nat ionai Center sought to itudy fully the effects bf 
cjassroom and teac|ier curricuiar differeo'^es within schools, it shbuld 
increase the number of classrooms selects.^ withiri i schbbl tb twb br 
three. J t should be noted that the practice of ' forming pseudb-schbbis 
as J composite of two or three classrooms from different schbbls or of 
forming pseudo-elassrooms of two or more smaller classrooms while 
.beneficiai for making riati^ wouid ribt be meaningful iri a 

study of classroom and school practices. If a country had a substantial 
riamber of smalj ruraj^^ s^^^^ than 25 students per class or 

with^less thari two c|asses^^[^ target population, a separate stratuifTi 
^ bf such schoojs cou|d. be fo schools to be 

iriclu^ied In the calculation of national estimates and in the arialysis of 
charige iri performarice over a school year. 
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it is increasingly common to find schools and classrooms organized 
on an open plan with fifty or more students m the classrodm group^ Jn 
such a situation it would c]early be red Uhdaht:.td_ collect information on 
student performance for more than 25 or 30 of these students, and some 
sampling of students within classrooms would be desirable. 



ii.a A Sampling Plan . " 

in the development of a sampllhg plan each National Center would : 
obtain a list of alL schools within the regions in which the study was 
being conducted. Thus in the United States it might be considered de- 
sirable to select certain school districts or certain States for involve- 
ment in the investigation, while in Australia it might be considered im- 
portant' to include all States ahd_ Territories, but to draw separate 
samples from within each State. The Msting:qf_ schools , would be stra- 
tified by appropriate schddl stratifying variables (e.g. , school type] , 
which from previous; research studies were knovyn to be related to 
student achieyemeht and teaching practice, in addition , ah estimate of 
the number of students within the target population should., if pbssiDle, 
be obtaihed fdr each school, together wjth an^ estimate of the average 
size of class so that the number of class groups within each school 
could be estimated. In the first stag^ of sampling schools would be 
randomly selected with a probability proportional to size, estimated in 
terms of either the number of students in the targe^ lDopuIation_withln 
the school or the number of class groups withm the school. (Strati- 
fication of schools by size will usuaUy be essential.} In the second 
stage of sampIin^,_one, two or three class groups would be randdmly 
selected from within each chosen school for participation in the study, 
again with a probability proportional to size._ In those classes within 
schools where th^re was an adequate degree of Uniformity in^_d^ 
It would be unnecessary to sample students frorti wlthin^Iassrooms^ 
provided between 25 to 30 students were present witnin each ciass. ]f, 
hdwever, c[assro6ms contained substantially rrpre than 25 to 30 students 
It would be more efficient to sample students frdm within cjassrooms. 

Since three, and in certain cases four, stages of sampling are en- 
visaged in this plan the design is necessarily a complex one and_any 
occurate estlmatlbh of sampling, error cahhdt be undertaken by formula 
but only by an examination of the data eoljectedjn the study. 
procedures are available for the calculation of samplmg errors, namely 
'balahced repeated replicatidhs' arid JJacknifing. ' Where it is necessary 
for a large number of strata to^/be formed it would be impor^ 
le^st two schools were drawn from each stratum so that balanced re- 
peated replieatibh prbcedures could be used. Where relatively few 
strata were included in the sampling, frame It would be importarit^ to 
have at least five schools drriwn from each stratum so that jackriifihg 
procedures cdUld be Used most efficiently. 

Alternative procedures are available for obtaining a sample , of 
schools arid classrooms within schools where selection Is made with „ a 
probability proportidnaI__tp size. The procedures depend on the avail- 
ibie information associated wjth the number of students and the number 
of classrooms within each schooh The selection of an apprbpripte pro- 
cedure should be made according to the nature of the informatibri most 



readn^^vanabie. Fdrther details of alternative prbeedures are gjveh in 
the Sampiing ManaaL (lEA [MATHS-N2]/A/14gj . It is, however, im^ 
portant that random selections are made at each stage of_ sam pling.. _ 

i1;5 Units of Analysis 

" The data which, are tb-be gathered for the study may be ahalj^se^^^ 
at different levels of aggregatibri. Table. 11 . 2 describes the level of 
aggregatibri for four jDbssible sburces of data ^students, teachers, 
classrooms,, schbbls) when applied to four ; pbssible units of analysis i 
(students, classrooms, schools, countries). 

Table 11.2 ' : . • 

Level bf Aggregation of Data for Different Units di Analysis 



Source of data _ ^ Unit of Analysis 



Students Classrooms Schools Countries 



Student data 


Same 


Aggregate 


Aggregate 


Aggregate 


Teacher data 


Disaggregate 


Same (Wtd) 


Aggregate 


Aggregate 


Classrbbm data 


Disaggregate 


Same 


Aggregate 


Aggregate 


School data 


Disaggregrte 


Disaggregate . 


. Same 


Aggregate 



Thfe development of descriptive profiles could be uhdertakeri by us- 
ing students as the unit of ahalysis for studeht data, arid classrooms as 
the unit of arialysjs for classrobm data arid teacKer data, Iri sample de- 
— sigt^T-wher^e^-mb^re-^ari-H^rie then 
the descriptive profiles for sehbbi data cbuld be uridertakeri by using 
schools as the uriit bf arialysis. , 

The cbrisid^ratiori of factors which explain change in. mathem atics 
performariee wilT focus bri the classrbbrti arid- the schobr a the Unit of 
arialysis. Iri some schools the teaching of a; class group is shared by 
two or rhbre teachers either within ah open plan or team teaching situa- 
tibri cir merely to maihtaih a balanced wdrkldad for each teacher. In 
such circumstances it will be necessary td cdmbihe the ihfdrmatidh cdm- 
pbsite of those teachers prdvidihg a learning experierice for the class- 
room grdup. The basis of weighting should be the tirrie spent with the 
classrddm group. 

11.6 Estimates of Error •\. 

When , using data from djffe rent sources, namely r students, teach- 
ers, classrooms and schools different leyels^ of error ar^e present for the 
differeht statistics likejy to be^^ 

mating, prior to th^e coJJecti^^ data, the errors which are involved^'in 
tFse presentation of the results of this study it is necessary tb. make 
guesses of the probable values of the design effects to be found; ; 
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Table 3 records estimates of the standard errors for several differ^^ 
statistics: prdpdrtidhs/ means, and regression cqefficients. in making 
— ^ese-^es t4ma t^g-w<^^ f o r s tu de rits 

withiri-schbols of 0.1 ^ for students within ciassrcxjms of 0,2, and aver- 
age sizes of school and classrodm groups of _p and 20' students respec- 
tively. These assumed values are consistent with estimates for these 
statistics made in earlier lEA studies, 'although^ c 

has been found between countries according to the nature of the school 
and curriculums existing within those countrlesi 

table 11.3 Estimates of Standard Error f ^ Riffe ceat Statistics Using 



;es 



Statistic : : 




Source of 


Data 




Students 


Classrooms 


Schools Countries 


Proportions 


0.02 


0.04 


0.05 , - 


Means^ 




0.08s^ 


O.IOs- 


Regression Coeffi- 
cients 


0.04 ; 


0.08 


0.10 


Recommended Number 


4000 , 


200 


100 1 


Simple Equivaient 
Sample 


800 


1 40 


100 ■ 


^sT'^e' ^s ^""^ 


standard 


deviations for 


students, classrooms 



and schools respectively. 



Slace the testing jDrogram will iTivblve the administration of a core ^ 
test with four rotated tests being admin istered to sub-samples of the . 
students some test items vyill 6e~taRIfr"by"^ 

reduction iiri the number df students taking certa fn it^ms vvi 11 onjy 
i h ch^ili rtia r^in^^^^^^ t he s tandVrd e r rors fo r plx)pd rtibns" and me values : 
retorded* in Table 3 for dat^ cdllected from student^^ all four 

rotated tests are admihistered ih each classroom. 

It shduid be noted that much pubj[shed^ resear^^ 
ployed regression coefficients whjch have maga[^^^^ much less than 
0.16. Consequently; it would seem that the use of only 100 schools, or 
a simple equivalent sample of only 140 classrooms would run the risk ^f 
vailing to detect as significant some^ potentially interesting school and 
classroom variables. For a full examinatjoh of school and classroom ef^ 
fects it would clearly be desirable to approach as near as financial re- 
sources would allow, the upper bound of samjDie sizes recommended in 
Table 11.1. . . 
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11i7 tevel of Response 



Daring the lEA Six Subject^ Survey , _whi^h was limited to the 
cbllecticin of d ata±at::jone^poini^in~~time-^^ in the^ sxecu- 

tioh of the sample design ^^ere such that; in 10 oat of 20 coantries the . 
response ratfe was less than 80 percent of the students in the designed 
sample arid iri seven of these ten eburitries the response rate was less 
than 60 percerit of the students in *the designed sample; Ther^forev it 
. is impbrtapt^'tb recognize that: In this study we are attempting an ex- 
tremely imbitibus data , gathering operation by cblleetirig data at two ' 
pbirits iri time, frbm^ in general , two or rribre classrobms in a school 
with a substaritial demand bri the teachers during the school year in. the 
CQmpletidn of questidririaires. While the sampling plari may tfe carefully 
desighed.and the level of resporise_ iri the initial stages bf the iriyestiga- 
tidh high, the impdrtarit eviderice for the study is beirig cbllected at the 
final stages and during the schbbi year. It will be riecessafy for 
efforts td be made td r^Uce the administrative Ibad bri schbbjs arid 
teachers in order td mairitairi as high a level bf respbrise as pbssible so 
that sound analyses of the data can be carried dut and useful generali- 
zations made. 

Commentary on the Sampling Plan : 

1.1 ;8 Facilitatihg National Analysis ^ 

It is likely th^J the Sec^^^^ effectively pre- 

empt the energy and reisoarces for high-quality International surveys iri 
mathematics for a long time^ certainly a^ d.ecade or more. It is even 
rribre important to keep J h mind that for many countries , participation in 
the ' Second Mathematics Study will absorb the local energy and /e- 
sburces for extensive riatibrial surveys in mathematics for ah equally 
Ibrig time. It cari be argued that, ultimately , most of the users and 
users of the study will be , for hatibhar analysis, interpretation and 
plaririirig. 

For riatibrial iriterests, therefore, the sample designs rriust be ar- 
ranged td facilitate withiri-cburitry analysis. This may be more Impor- 

„_tarit__tJteFL^p:cby 

Two aspects of the samplirig prbcess are eritiear to facilitatihg hatibhal 

First, hatidnal ceritr^s rieed^^t directed, ericbu raged, and 

Assisted in determining, substantively relevarit Itiajdr stratificatibri for 
their studies. Certainly in all countries there are ihlpdrtarit regibris, 
. types of communities, types df schddls;* or sUbpbpulatibris bf students 
or teachers fpr^ w^^ important educatidnai pdlicy qUestipris 

need to be answered i A process at the natidnal level bf discUssibri arid 
debate needs to be carried dU the critical strata arci 

|h one country these may be regional, while in another they may be 
: according to school stream. 
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IliS Disjirbportibhate Samplin A National Concern 

While sampiihg proportional to size (ppsj y[elds probability^ 
at the riatibrial level, this m^hdd may be disadv^ntageoas to some couh-- 
tH(§s. One cannot assume that sub populations of criticaJ national jri- 
terest (sUch as mihority ethnic groups or experimental technical 
— schoolsr^re present in equal proportions, ^ 

In all practical cases, national averages will be sufficSeritly ac- 
curate for- internationai or longitudinal comparisons hb matter what, 
scheme is used withjn country^ Bat if prbpbrtiohal sanrijDlirig is Used, 
small but critical groups of schools^ teachers, land students wUI likely 
ribt be represented iufficientl^^ in a sample to allow separate analysis, 
and that could diminish national interest in analysis. 

The prospect of disprbpbrtiohate sampjihg could mf ah reduced na- 
tiorial accuracy, but if safficieht accuracy is bbtamed within each major 
st/atum. It win b^e obtained over^ as well. .There will b)e cost, per- 
haps^ In incriased sample sizes, aad the paybff will be [ri imprbyements . 
just in national analyses. All national and international analyses will 
require weighting, since samples will ribt be self-adjusting. This means 
that the central data processing and analysis and the subsequent na^ y 
tional work will have tb jD*^bgram in weights, this does not seem to bey 
an overwhelming problem arid, in fact, the. necessity of using weightmg'^ 
means that other samplirig methbdblbgies ; such as some kinds of class- 
room sampllrig, ..Which would also require, weighting, , need not/ be t 
avoided. . . . ; 

11.10 Practical Error Analysis • . /"^ 

fhS analysis methqdojogies proposed for the Second Mathematjcs 
Survey are rtidre: sophisticated and complicated than in earlier studies. 
Also, it is planned that a x^riety of complex analyses will bg carried 
but and reported frorn data. It should be hoped that the complexity 
-^d^>>aj:jaty.,^ for realistic Iriterria tibnal arialysis will 
be made available for national analysis as well. \ — ^ ^ — \ — — 

The notion of applying cFassical error arialysi? prbcedures to such 
a variety of fancy analytic results in such a large riumber of riatibrial— 
and international a rcumstances a rider ebridi tibris bf very cbmpJex sampl- 
ing is obviously Jmpj^actical. WhatMs heeded is ^ generjal methbdbldgy 
for error analysis wM^ applied immediat&y arid fcbrisistently to 

aU analytic results for the surveys^ _ . J ■ 

the national sample desigris shbuld be arranged with ,,prope 
lancing and replications;^ to j^ljow interperietratirig sUbsamples so _that 
jackknife or balariced-replicate samplirig formulas can be universaljy 
applied to statistical results. Perhaps, the US-NAEP rule shpuld be 
adopted: no statistic should go out without having an hdriest^^e^^^ 
of its accuracy. The advantage, bf course, of subsarh 
procedures js, that reasbriable estimates of standard . errors .can be 
derived by simply repeating analyses bri subsampies. incremental 
computer costs [relatiN/e tb set-Up costs) are minor. , ■ ^ 
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There is a heed for explicU direction _ and gqidelines to national 
-cehtres to ehsure replication arrangements wh^^^^ _sampiihg 
proposals are reviewed. It will also be^jmpq^rtant to e^^ good 
— repircatibh-arrahgemehts wheh hati^ proposals iare reviewed. 

11.11 • School Size and Samplihg Glassrbbms " 



^ THe Cross-Sectibhai—Study to some extent and the bongitudinal 
^Stlidy fundamental ly are cbhcerhed with what happens in the vmathemat- 
ics classroom arid how that affects achievement. This means that the 
samplirig bf classroom's arid the lirikage of students and teachers are 
critical aspects of th^ desigri. 

. \ is. impossible to deal realistically with samplirig classrbbms [arid 
therefore students or schools) without takirig iritb accbuht the fact that 
In many (most?) schboj systems, studerits are assigried to classrooms 
expj id Wy accord in mathematics achieVemehts and interests and 

according to the instruction they are to receive. A samplirig plan 
vyhich dbes not take into account the streamirig of studerits by ability or 
the dif^mrent classfborti of cbriterit of iristructibri is gbirig tb 

suffer terribly In terms of accuracy and potential for analysis. 

T\fie issue is c^onniE;ct of sampling accbrdirig to schboL 

. size. Sm^n s^^^^ likely to have different numbers and kinds bf 

classroom^ aVrarig^^^ Perhaps thi^ suggests that sVatificatipri by 

size is desirable for subst^antiye reasm^^ any case, stratificatibri by 

size will be\ necessary for the technical reason of making rational, 
careful selectmhs of classrooms. 

It IS deSJrable that schools first be stratified by (of the 

populatiori bf mterest, of course). Within sjze strata, there seems to 
be rib reasohioJe alternative tb careful and^ planned sejection of class- 
rooins accbrdirig to the kirids bf classroorrrs whjch^appear. For the 
Lprtgitudinal Study certairily arid for any_part of the Cross-Sectional 
Study which presum^^ tb relate iristPuctloh to achievement, classrodms 
rather thah studerits must be sampled. But if there are, say, two low 
. a nd_ twb high strums iri a schbbi, the samp le must include one of each 
^Kinq^^— =^f=^ere— vrere^ law arid u ne-^gfrT-H:-!^— saff^^ till 
require at least one of each kirid. . 

_ f^'^^ dbvibuslyycqmplicates matters. , , The sampling and admihistra- 
tion plans will have^ to cdntemplate further Irifbrmatlbri: collection, more 
extensive decision processes, arid perhaps some iri-fie!d samplirig. The 
data \col lection, processing^ and analysis may have to provide for in- 
school\ weighting; \ 

11.12 Selecting the Longitudinal Part of the Sample 

Sne major varjab^^^ in the study is "growth iri mathematics" over 
the year .J^^ A^^ obviously requi res ^ represeritative samjpres 

of A^cMracy. the hypothesis that relatidrial studies of 
variables can bev carj-[eq on Judgment sampies is certairily^ restricted 
to very good judgmen,ts oj- very special relationships. Preferably, 
probability samples Will be employed, however. 



EKLC 



60 



the ideai approach to selection for the iongjtudinai s 
dom sampling. CarefuU substantive stratification, probably with^ls- 
propbrtibriate sarnplihg rates^ should be encouraged. If genera^ 
ing is not feasible, there are a number of alternat[ves with^but eliml^ 
ing the randdmizatidn. First, the scope of the survey . coujd be re- 
stricted^ fdr exampio^ by taking a randgm sampje wjthjh a convenient 
region or set of regions. Second, only schools of certain types^ suc^ 
as public schodis, might be chdsen. If nqn-random samples become 
. absdtutely necessary—tha^ Is, after attempts at drawing random samples 
fail — then it will be essential that the best noh-rahdom procedures, be 
empioyed. For example, an explicit iquota system might be ^used after 
fine-grain stratification. In this^^ase, the national centres are gdihg to 
need more guidance and assistance in defining the sarrifDlirig. 

11.13 Sampling pf Teachers 

Jj:ie sampHng plan calls for a national probability sample of 

vciasses. However^ this does hot aatdrnatically yield a cbheurreht sample 
of teachers associated with these classes which, is raridbm; I In par- 
ticular, the sample will be biased in the fqvbr of thbse teachers whose 
teaching assighmeht includes classes in the target pbjDulatibh. It is 
important tb keep in mirid^ therefore, that the data associated with the 
: sampled classes is a prbbability sample of the iristrUctibria! enyirbnmeht , 
in which the national jDrbbability sample of students is embedded. It 
might j3e_ founds for Jxample, that 30 percent bf the teaching df ad dj- 
tibh of fractions at_ Pbpulatibri A involved the use bf number line. In 
aatibhal profiles, of teachers (asj op pbs^^ tb teaching) were desired 
iWhat percent of teachers at the Population A level Use the number line 
in teaching additibn bf fractibps?) weighting, would need to be em- 
pioyed. The schbbi questibnriaires wiJI prbvide the needed informatidh 
for the weighting. ^ . ! 
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12, Structure ^of the cognitive tests 

The Cog hit|ye test^^^ nd_cons-tra Ints — 

For both populations^ the demands include 

12.1 Cross-national curricular va — 

Meahih q ful com pa ris6ris_, across countries require measures which 
have validity in the countries involved. The . effort made here is the 
familiar one of devising a test which Is, as far as possible, eqaaljy 
"fair" or "Unfair" tol al I _cbuhtnes, with their individual programs and 
edUcatibrial ^bals. See Peaker [1569, pages 229-237) for a techhicat 
discussibri bf this issue, vvith specific reference td First Study. ^ 

The fbl towing procedures were follbwed in developing cognitive in- 
strumehts which meet the curricular validity criterion. (These com- 
ments are drawn from a memb prepared by Robert Garden^) 

12.1.1 The Interhatibhal Grid 

In responding to Working Paper I, Natibnal Centers indicated the 
relative impor^tance in the mat curriculum bf their countries of 

each topic and behavior on ^comprehensive lists for each pdpulatibri. 
From these responses international Grids were constructed in which the 
elements common to most countries' curricula included. These 

grids, later modified in response to further replies^ to Working Paper 1, 
fornied the basis for blueprints for drawing up test forms. (See Tables 
12.1 and 12.2;) It should be noteduthat^the grid^ irhpbrt- 
arice with which Nationaj Centers regarded topics and behaviors for the 
Population A and B leVel: students whether' they taught at those br St 
earlier levels of schooling. 

12.1 .2_ The Item Podls 

Trial items were drawn jfrom: 

(1) items Used in the First lEA Mathematics Survey " ' " - - 

(ii) items sent by couhtries proposing to take part in the study 

(iii) items constructed from tests, item banks and examinations 
sent by cbjjritries^^^ ' 

(iv) items cbristruct^d for thbse cells in the grid for which items 
were ribt available frbm other sources. — 

In assembling the pbbis bf items the major factbrs taken into ac- 
count were: / 

(i) the need to sample the jriterriatibrial Grid adequately with 
items as free as possible_ frbm cUltUrar bias (e.g., currency) 

(ii) the need to in dude "arichbr" items from the First. 
I EA Mathematics Survey to allow for comparisbris tb be made 

, where this seemed appropriate 

(iii) the need tb include ite^^ oh particular fields of interest of 
the International Mathematics Comrnittee, ♦ 

For jsxsmple, the Cbinmit^^^^ that items including 

calculations with mone^^ be included for Population A, even thbugh this 
w^uW cause some translations problems in one, or two countries.. Items 
appropriate for hand calculators were also included. 
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... The dassificatidn of items by behavior is regarded only as- sug- 
gestive of levels of cognitive comjD^lexity. Some Items will iiave a dif- 

4ei:ent^class[^^ 

as prior instruction. ; 
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TABLE 12,1 



Popoiation A ; 



IfflP-OXtarice^Eor^^ nstPument-8ons^truction 
Of Content Topics Arid Behavioral Categories 



Cbriterit Topics 



ixxrai Catego ries 
Computation 

eompreHension 
AppjjMtidh 
Analysis 



000 



Arithmetic 

001 Natural numbers and whole num 

002 Cdmmbh fractions . ; . ... 

003 Decimal fractions. . . . ... 

004 Ratio prppbrtibh , percentage . 

005 Number thebry . . . . i . . . 

006 Powers and exponents . . i . . 

007 Other numeratibh systems . . . 

008 Square rbbts ......... 

009 Dimensibhal analysis. ..... 



... 


V 


V 




V 


V 


... 


\^ 


V 


. . . -. 


V 


V 



V 

I 

V 



106 



Algebra 

101 Integers . . . 

Ratjonals . ... . . . . ...... 

Integer ej<:pdnents . . . . ... . . . 

104 Formulas and algebraic expressions . . 

1 05 Polynomials and rational expressions . . 
Equatbns and inequations Clinear only? 
Relations and fu net idhs . . . . . i . . 

Systems of Ijnear equations ...... 

Finite systems. ............ 

Finite sets . 

Elbweharts and programming . . . ... 

Real numbers . 



102 
103 



106 
107 
108 
109 
110 
111 
112 



200 Ueomet r y 
201 Classi 



. iL^ajDje^ figures-- . 

202 Prbpert|es of plane figures . . 

203 Cbhgruerice of plane figures. . 

204 Similarity of plane figures . ; . 
2P5o Geometric constructions • . . ^ 

206 Pythagorean triangles . . . . . 

207 Cqprdinates . . . . , . . . . 

208 Simple dedUctibhs . . . , . . . 





Y - 


—r 






V 


1 


jS 




J 
1 


J 
1 


jS 




1 


1 


_s 


• s 




|S 




jS 


|S, 


1^ 


L 



TheTollowing rating scale has been used: V = Very important; I = impdrtaht; 
Ig =»important for some countries. A dash (-y iRdlcates that the topic was not' 
considered important enough to warrant trial items being found of constructed . 
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TABLE 12.1 (Continued) 



^ ehavior c 



Content* Topics 



Computation \ 
eompreHerisibh 
Appllcatlbh 
Artaiysis 



200 



Geometry (Corjt-d.) . ^ _ 

209 Informai transformations in geometry, . • 
Relationships between lihes and. planes ih' 

S pace • • • • • • •_» « • • • m - m 

Solids {symmetry prbperti^es)^ . ^ . . * . 
Spatial visual izatlort and represeritatioh 

Orientation [spatial]. . ... i .... . 

Decomposition of ftgures. . ; . . i i . . 

Trarisformatibhal geometry. . 



210 

211 
212 
243 

2ia 
215. 



360 Probability and statistics :^ 

301 Data cotlectibh. . . ^. ........ . 1'^ I 

302 Organization of data. . . . . • . . . . . • ! l 

303 Represehtaiibh of data . ..... .... I I 

304 Ihterpretatidh of data (meari> median > mode) I I 

305 Cdmbihatbrics . . . . . .......... 

305 Outcomes^ Sample spaces and events. ... I- ^ 

307 Cpuriting bf sets, {PJA B)> P(A B), 
ihdejDerident events ............ - - 

308 Mutually exclusive events - 

309 Complementary events. ^ 

400 Measurement . « 

>01 Standard units of measure. ........ V V 

402 Estimation , . . ... ^ ........ f . ! ' 1 

403 Approximation ........ ^ .^^ ^1 I 

404 Determination of measures: areas, volujnes, 

^ .etci.^--^.:::^^::.^^^ ^ 



6:5- 



TABLH 12.2 



Population B_: importance For Instrument Gbnstraction 
Of Content Topics and Behavioral Gategdries 



Content Topics 



Behavioral 



Computation 

CbrrifDrehehsion 
Applieatibri 
Analysis 



1.2 
1.3 

1.5 



Sets> relations and functions 
lil Set nQtitiph . . ^ J ^ . 

Set ope rations (e.g., union, inclusion) . 
Relations . . . . . . .......... 

Fuhctibhs ._ . 

Infinite sets^ eardihality and cardinal alge- 
bra (ratibhals; and reals] ...... ^ . . 



V 



V 



Number, systems _ 
2.1 Cdmmbh laws for number systems 
Natural numbers . . . i -. . . . 

Pecinials ........ ^ . 

Real numbers . , 
Complex numbers 



2.2 
2.3 
2. a 
2.5 



I 
I 

V 



Algebra 

3.1 Polynomials (over 

3.2 Quotients df_ poly ndmf . . . • 

3.3 coots and radicals . . . . . . . . 

3 4 Equations and inequalities . * . . . 

3.5 System of equations and Vnequaiities 

3.6 Matrices. .rT. 

Jul Groups, rings and fields > . , 



V 
V 

y 



V 
I 

V 
V 

y 



V 



y 

V 



s 



Cebmetry - 

4.1 Euclidean {syhtjieticj geometry ...... 

4.2 Affihe and p^•ojective geometry in the plane 

4.3 /Arja|ytio (cbbrdihatej gebmetry lj>^the plane 
4,.4 /Three-dirriensibhal cbordiriate geometry. . . 
t.5 Vector methbds . . ............ 

4.6 TrigdndrYietry . ^ . . . 



y 



y 



y 



The following rating scale has been used: y = very important; . I = irripbrtant; 
= important for sotrie countnes. - A dash (^) indicates that the topic was not 
qdnsfdered impbrtant enough to warrant trial items being fdUhd or cbristructed . 

This. sectloR is currently being rnddified to take into account curNcular empha- 
ses In sdrtle European cbuntrfesi , 
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- tA RJ^EJ-2^-2^(.Contiade.dX 



ebnteht Topics 



Geometry ( Gont'd . ) 

4.7 Finite geometnea . , 

4.8 Elements of topology 



Behavioral Categories 
Computation ■_ ; ; • 
Comprehehsibh 
licatibn 
Analysis 



Analysis _ 

5.1 Elementary fijnctibhs • . . 

5.2 Properties of furictibhs . . 

5.3 Limits arid cbhtiriUity: . . . 

5.4 Differeritiatibri 1." , . f . . , 

5.5 Applicatibris of the derivative 

5.6 integratibh .... . . . . 

5.7 Techriicjues of ihtegratidn . 

5.8 Appiicitibris bf integratidri. 

5.9_ bifferehtiai equations . . . 

5.10 Sequences and series of functions 



Probability and statistics 
6.1 
6.2 
6.3 
6.4 
6.5 



Probability . ... 
Statistics_.__. . . . 
bjstributions ... 
Statistical Inferehee 
Biva riate statistics 



V 


V 


V 


V 


V 


V 


V 


1 


1 . 


1 


1 




V 


V 


j 


1 


V 


V 


V 


1 


V 


V 


V 


1 


V 


V 


1 


1 


V 


V 


V 


I 


1 

s 


1 

s. 


1 

s 


1 

s 



V 



V 



7 


Finite mathematics 


. . . . . 1 


1 1 - 


8 




...... 1 

s 


1 1 - 

s ■ 


9 
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12.1.3 Item Trials ^ _ : _ ' 

:/ ; From \he assem 10 collections of 40^ at Popd- 

I^tidh A level arid 13 collectibhs of 26 items each at Popdlation B level 
were field tested in several countries. In one country a further 2 
collectidns of 90 items at Population A level were field tested. The 
Committee examined the item statistics and comments by Natibha!. Geriters 
resujtihg from this fiejd testing, paying particular attention to mathe- 
matical , aj^prdpriateness, psychdmetHt properties and cultural bias; 
earefui interpretatidn -was necessary as comparatively few countries had 
taken j^arj: in the ti^ some had admihistered the items before the 

end of the. school year ^ '^r with students, at higher gJ^^de ?evei than 
Population A. Inevitablfy, Sdrrie measure of cdmprbmi'^e 'was U 
In meeting the needs df wrIiSely differing curricu?ums and instructional 
practices. : 

Further ite^ wei^e consiructed for tells In the Internatidrial Grids 
^tb supplement the pools and replace deleted items* The Iriterriatibrial 
Grids were modified sMghtly as a result of additidfral replied received 
from cotihtr-ies in response to Working Paper j. The next round of trial 
testing included 12 collections of 40 Items each at Population A level and 
15 collections of jo items each at Population. B level. The IMC has 
selected,: thiose items which appear to be most appropriate fdr the pur- 
poses of the study. " " * 

At the lEA General Assembly In Paris, September 1979 / a j^e^^^^ 
to the IMC was submitted tb the IMC in behalf of several countries, 
in clUdiJig French and Flemish' Belgium, France and tuxim bod rg conce 
ing the lack of items whjch accurately reflected the distinctive nature dj" 
their curricula. The shbrtfall, it was noted , was particdiariy in 
gedmetry. As the result bf several fneetihgs with various members of 
the natidh^l cdmmittees, arid further field testing in some codntries. It 
was agreed to further modify the iriterhatibriaf grid arid to iricremerit 
the intern3tidnal tests. 

Jgopulation A , . 

Mcdification of Grid: 1 

Add ^ to geometry. Section 200, ttie category 215: Transformational 
Geometry. These are to be 1^ cells at behavioral levels I - III. 

Items to be Added to Test 

Eight items were prodaced, 2 for each of the four rotated for m:^, as^— - 
follows: : 

: : Behavioral Number 



Cbhteht bevel of items 

103 11 " 1 

205.3 ^ III 1 

207 II 2 

215 J 1 

215 II 2 

215 . Ill _ i 1 

Total ~S ~ 
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PbpUjatidn 8 ^ 

Mbdificatioh of Cnd ^ . - . -. - ^ 

Categdries wilT have to be added or present categdries slightly rede- 
fined, especially in Section 4, Geometry, to take account of the special 
characteristics of the curricalam in the countries requesting this modi- 
fication, ' 

items to be^ Added to Test - _ _ - - ^ 

Sixteen Items were produced, two for each of the eight rotated forms i 
Since , the grid is not yet revised to take the new subject mattery into 
account, a precise tabalatioh is hot available. The items are primarily 
in geometry; . - c.; : 

12.1.4 Anchor Items _ ^ _ L 

Ah objective of the Second Study of interest to many of the counr 
tries which participated in the First Study is. that pf examining differ- 
ences in s^tudent outcomes since 1964, the date of the First Study. 
Evidence of such differences will be gained through the u^^^^^ of anchor 
items, that is, items used in both the First and Second Study. 

For the cognitive instruments^, the : aehor items raised special 
problem areas. The selection of items from the First Matheme^lics Study 
pool was difficult duq to amb[guity - in how the First Study jtem mea- 
sured the stated obi^ectives. It was also impbrtarit to select items which* 
would help address issues in mathematics edueatlbh such as what 
changes in computatioiiai abiiity have occurred since the First Study. 



Study does not guarantee meahihgful comparisdriv What is desired in 
measuring changes In student performahces over a span of years is a 
measure of the same behaviors at the twd pdihts of time. However, 
schools- and curriculums change. The instrUctidnal cohtent changes. 
Hence, the IMG strove to take into accdUht changes Jn usage, cdnyen- 
tion, terminology; and other factors, in attempts to prov^ide measures of 
the same cohstruct. 

The result , of the^e efforts is three categorjes of anehdr Item^^^ 
One categoryjis that of items which have >ema]ned identical from ^the 
first study. The second categdrj/' j^ a set of items which have, changed 
in terms of format, e.g. , open-en^ed to multiple choice. ^ Tine third 
category is a "middle group" of items whjch have undergone onjy minor 
changes in wdrding, and in the judgment of the IMC, samjDie for all 
-pFacticaUpUrpdses_the!^same--.cons-tr.uctsJn„th^^ 

A summary of the anchor items a^^ in the cognitive tests for 

the crdss-sectlonaj study Is given In Appendix C. 

12.2 Responsiveness to growth during academic year (Population A) 

The longitudjna[ aspect of the Study requires as a dependent mea- 
sure growth in those aspects of mathematics being examined in -th^ 
classroom process questionnaires. This is indeed a stringent require- 

. . 72- 
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merits fo r it g oes beydrid the validity issue to harrowihg the curricular 
emphasis to a ebmrribn period of time crbss-hatibhally (that is> between 
the admiriistratibri of the pretest and fDOsttest.^ 

The initial atterrifDt blF the . Ihterhatibrial Mathematics Cbmmittee tb 
prbidace. sach a iheasdre was the prbpbsed corrimbri core bf items tb 
S'^^-'e as the growth meajures (5 scales of 8 Items each) ._ The res^ults 
of the pilot testing for Populatibh A are summarized in Tabjes 12. 3 
oages) and 1 2.^* (5 pages) . The median difficulty "of the items is, for 
most cdUhtries> Within acceptable ranges for the end of the year. 
Japan and Hdhg Kdrig are notable exceptidhs in that the items ^re very 
easy fdr thdse cbuntries. 

Countries were also asked to rate the items according to when they 
taught . m th:i 5chdol J^ear Qnipprtant infpf nvation for Jthe_ class ro^^ 
process qu3St]qnnai»^e) and the appropriateness of the items to the tar- 
get populations. Table 1 2.5 (3 pages) summarizes /this_, info 
Clearly, adjustme^nts in the c^ tests are needed/^o rthat they better fit 
^t^®_i^PTrJ?yl^Q^^s J Much of the remainder of this section 
is addressed to this issue. 

Two other analyses were done , on the pilot data tb helfD shed light 
upon the current structure of the Population A test. Table 12.6 shows 
scatterplbts bf between cbuhtry corrdatibns fbr various topics. New 
Zealand is chbseri as the 'vertical scale since rnbre bf the items ^were 
pilot tested there than in bther cbuhtries; Table 12;7 illustrates the 
relatibriship between the core and rotated form subtests by dountry 
median difficulty. 



Table 12.8 depicts the jDetweeh schdbl differences oh the core items 
fbr the United State and Canada. With the exceptibh bf a few items 
that were tried tw^^^^ with two different samples (in Jajiah 

aiid New Zealand) , this represents; the bhly ihfbrmatibh available tb the 
ImC abdut within country variation at levels other than between, stu- 
dents. Th^ between schddl yariatipH fbuild iri the ij;S. arid Canada 
may_ridt be representatiye df what will be fdurid in the majdrity of 
countries. If not, then this table rtiay be disregarded. But if so, this 
table may cdntaiii impdrtant informatidn fdr thdse whd are structuring 
the cognitive instrurTients. 

° The majority of the discussion of targeting the difficulty of the 

cognitive test has had as a context the jDnoblem oT between c^^^ 
differences. ^ Concerns have been expressed that the cognitive items 
may be too e^asy for one or two of the cquh and too d[fficult for a 
few . of the others. Seldom has the notion^ of looking at withm country 

differences -ehtered__these— discuss But, — if— there— are— substantial 

within cbuhtry differences at the schbol or classroom level ah overall 
adjustment may not be the most efficient. Since the focus of the study 
is__bn the classrobrfi, an aim shbuld be tb eliminate ceiling arid flbbr 
effects at those levels. It may be riecessary, therefore, to get esti- 
mates df within cpuhtry-. differences at, at least, the schobi level, so 
that a cognitive iristruirierit cari be structured that is varied eribugh tb 
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was aii in ail encduragiri§> it is clear obtaihihg suffie^^^^ variation in 
the. measures to yield interesting analysis wil| require scales which are 
rrior^ rejponsive to t^^^ currieulurns. What follows is a de- 

scriptjon of suggested structural changes for;the cbgriitive test arid 
procedures for allowing countries to construct the best possible cogrii- 
tive test for their purposes. 
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TABLE 12^3. ItemJDif ficulties by Cbuhtiy for Pbpiilatidh A - Cbgriitive. Test. 
(Jiihe 1979 version) 



TOPIC' , 



' FRAGTIdNS- 



RATIO , PROPORTION , ' PERCENT 





1 


2 


3' 


■ k 


5 


■ 6 


T- 






■• ' i 


2- 


3: 






■■ 6 


7 X 


.Beigimn (Fl) "67 


. 87. 


75 








85 


70 • 


Belgium (Fl) 


56 


71- 


kl 


■ 


.71 




81 ,80^ ■ ' 


Beiffimn (Fr) 




.31., 




.'60 




50 


k6 


Belgium (Fr). 


39 


51 


10 


38 




16 - 




Can/USA 




79 ' 

1 ^ 




Ik 


63.' 






• Can/USA 








5k 


73 


1+0 


•• 72 










57 




.38 






France'.. 
















Hong Kong • 






85. 


.97 










Hbhg Kbhg 








11 






95 " . 


' Hungary 




■ 63 




,60 


57 






. Hiihgary ' ^ 


■ • 








35 


.37 . 


■61 


Ireland' 




57 ■ 














Ireland 
















Japan 




'93 




.'95 


. 83' 


90 ' 


93 


65. ' 


Japan , ' 


57 


61 


57 


6i 


73 


; 81+ 


81+ 89 


Netherlands ^ 


6k 


76. 


k6 ..' 


85 


62 


69 


. 91/ 


Ik 


Netherlands 


if9. 


■ 52 


28 


5iJ- 


. 61+ 


73 77 


N. Zealand 


3 k 


7i 


53 


62 










N. ' Zealand 
















Scotland . 








77. 




^72 






Scotlaind 
















Sweden 


















• Sweden • 
















Median 


6k 


76 


53 


77 


61 


66 


O Q 

oo 


60, 


■ • Median . ■ 


53 


57 


3^ 


58 


71 


ho 


83 77 


ALGEBRA 


















GEOMETRY 


















. i 


2 


•J 


. k 




' 6 


7 


8 




I 


2 




Ij 




6 


7 8- 


Belgium (Fl) 


73 










56 


ItO 


57. 


; Belgium (Fl) 


78 


■k6 




36 






5k '■■'<. ■ 


-Belgium (Fr) 




91 










1*2- 




BelgiuSn (Fr) 


78 


k2 




11 






Ik 


Can/USA 


k9 


.27 


38 




33 






• 31' ■ 


. Can /USA 










kk 




hi 22 


France 


39 


61. 


■62', 




27' 






37 


France : ." . 










Q 




53 ■■ 


Horig Kong 






88 


76 










. Hong Kong 




'2k 


96 


1*1 






63 . 


Hungary 








23 


ko 


33 






Hungary 






51 




59 






Ireland 




.61 






52 








Ireland . 










52 






Japan 








'81; 


Bk 


70 


63 




Japan 


91* 




9^ 




Sk 




81+' 


.Netherlands 


6 k 


6o.' 


79 


'57.. 


55 


37' 


29. 




' Netherlands 


73 


1*6 


79 


27 


5k 


•65' 


66 23 


N. Zealand 




36 


63 


21 




kl. 






N, Zealand 




31 


65 


'29 




77 . 


67. 35, 


Scotland 




63 






52 






.1+0 


> Scotland : 
















Sweden 


















Sweden 
















Median ' ; 


57 


61 , 


63 


57' 


52 


:k:i 






Medijgl__-.i-^-- — 


-7812 


79 


29 


5k 


65 


5k 29 





































"MEASUREMENT. 



• Belgiiim (Fl.) "86 
' Belgium (:Fr) 
. Can/USA, ' , . 52 

France 7^ 

Hong. Kong. 

Hungaiy 
.'^ Ireland ■ 



32 
33 



5' 



.8 



Bk .1+7 26' 70 i33 
U2 81/ 83 27 



Netherlands 
N. Zealand 
' Scotland ' 
Sweden . 



. 6k 80 57 97 . 72 k3 89 

23 

'5X ; : . 

80" 83 . - 93 

92, k9 59 . k2: 81 69 . 37 " 7k 

6k k2' 51 37 60 65 ko ka 
.83 . .58. . • 

79 ¥0 .6k k2 81 .70 37 ; ik 
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TABLE 12. 3 - Continued; 



ROTATED ^FORHS 



RATIO. PROPORTION, P^ GENT- 



. - {Fll ,T2 62 : 90 

(Fr). '67 70 . .. 
Can/USA $6 33 
France • 39 36. ... 

87 ^ ' 91 96- 



■Irelahd * * 58 

Japan _ . - 86 82 

N^therlkhds • • . ^ ' __ 63 . 67 

N. -Zealand , kg 50 ; 35 '38 59 

Scotland ' . . ^ 69 60. 

Sweden ' 65 ' 56 ' 13 ■ 6 5- 

Median 67 56 35 '5^' 75 

MEASUREMENT • . 







- ^ 








-6- 














4P- 




-2X- 


-i5- 


: ' (FlJ 


;83 




33 


Its 


,87 


76 




■89 




30 


: 55 




60 






lt6 


_ (Fr) 




73 




■k9 




71 


16 






■13 






38., 




22 


23 


. Can/USA 


85 








82 








27 












28' 




France 


















31 
















Hong Kong ' 




.80 




k2. 






38 


51 














66 




Hungary 


kg 








15 








52 










It 






Ireland 


























36 : , 




25 


• 25' ■ 


Japan 


96 


<? 


72 




93 


79 




■96 


79 , 


■68 








93- 


-75 
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27 


^0 
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71 
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36 
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Hong Kong 




89 
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77 


51 






21 
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35 
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15 
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39 
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96 
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92 
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85 
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TABLE 12.3 Continued 




REMAINING ITEMS ON ROTATED FORMS 



TOPIC 

IfflOLE NUMBERS 





1 


2 


/3 




• 5 


6 


7 


8 


9 


^ 10 


Belgium; (Fi) 


32 






9i 








75 


92 


61+ 


Belgium (Fr) 




86 








82 






86 




Can/USA ■ 


29 


82 . 


92 




'86v 


72 




58 




1+1 


France 




















56 


Hong Kong ' 




89 




97 






79 








Hungary 














62 








ir eland 




Ok 






6d 


73^ 






60 




Japan 


re 


93 




93 




81+ 


81 








Netherlands 


90 




19 




66 












N. Zealand 


23 


86 


86 


73 


8i 


81 


■51' 


66 


81 


71 


Scotland 
















83 






Sweden 




78 


67 


10 


U3 


77 


7lt 


.62 






Median . , 


32. 


86 


86 


91/ 


63- 


81: 


7U 


71 


8if ! 


60 



NUMBERS (OTHER) 
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Belgium' (Fl) 






61 


76 




73 




Belgium' (Fr) 




. 57 


16 




79 






Can /USA 


66 


57 




1+2 








France 


61; 






66 








Hong Kong 




81 


36 




86 


75 


61 


Hungary 


28 












21 


Ireland . • 




63 






75 






Japan ; 


87 


7if 






72. 


96 


67 


Netherlands 






-55 






63 


59 


N. Zealand 




58 


17 


53 ■ 




26 


21 , 


Scotland . 








1+8 


77 






Sweden 






-M 




10 




-16- 


Median 


61 


58 


1+1 


53 


75 


68 


1+0 " 



PROBABILITY AND STATISTICS 



Beigimn (Fl) 

Belgium (Fr); 

Can/USA 

France 

Hong Kbhg . 

Htingarjf 

Ireland 



J__2_ 



65 77 62 

7 30 31+ 

30 ■:27 , 72 1+0 71 39 
23 25. . ' 

1+0 1+0 : • 87 



36 .67 

32 r-5 



k9 



16 



Netherlands 
N. Zealand 
Scotland 
Sweden 
Median 



■ 35 19 33 2 
• 28 6 

1+9 70 .9Z. .83 83 8i 

36 37 

36 28 . 67 kh 6k 1+7 61 ■ 

1+9. 33 : - 

1+5' 67 kB- lU 35 h5 66 76 54 ' 

■1+0 33. 66 1+6 61+ 37 66 71 67 



92 
86 

7l_ 



79 



Belaium Flemish, 
Belgium Frenc)^ 
eanaclaiU.S/ 




CORE 
fofitenis M 



1} 
2 
2 
3 



72 



6S 
^7 
91 



57 

■ » 

91 
58 
75 



Range 
67-85 

31-87 
60-79 
38-57 
85-97 
57-63 



65-95 
53-71 
72-77 



ROTHED FO RMS ' 
Med Range" 



23 



9 . 


69. 


:37-85 


11 


70 . 


: W3 : 




■ 57 :' 




10 


47. . 


,'33-90 


12 


76 


51-95 ■ 


«i 


67^ 


.12-77 , 


5 .. 


56. 


22-66 


1i< 


8( 


63-96 • 


19 


52 


11-71 


9 


■ 7r 


k- 

35-97 



Table 12. if (Cont'd.) 

(PERCENT, RATIO, F»ROPORTibNj 



# of Items 



C b R E- 
Med 



RO;TATED FORMS 




Tdtal Pos'-jbl4 



12 



83 
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Belgium Flemish; 
Belgium French 
Cahada/iJ.S. 
France 
fIbngKbhg 
Hungary • 
Ireland 
Japan 

New Zealand 
Scotland 

Total Possible 



(ALGEBRA), 





r (\ 0 c 

L U K t 




i of items 


ted ; 






57 


10-73 




66 


■ 12-91 


6 


32 ; 


;6-il9 




39 


27-62 


2 


82 




3 


33 




2 


52 




'£1 ^ 


75 


63-8i| 


J 


47 


21-63 


3 


52 


W3 



'_' _ i ■ 
_ R 0 f A T E D F 0 R M S 

: l of items Med Range 

'■ ■ IS, 80 17-93 

13 65 31-92 

13 17 20-75 

7 53 • 17-79^ 
1 0 83 61-91 

8 18 31-91 ■ M 
i , 11 32-76 ■ 

■ i7' 88 61-95 

21 16 11-78 

^ 63 12-83 

. 28 
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Table i2.i| (»d.) ' : : 
, IMEASUREMENT) 

■ CORE ROT AT ED, FO RMS 

t of items Med Riikjc ' /iiijems' • K ' -Rajge 



Belgium Flemish ■ ' : . j : , 55 ■ 26-86 „ , 12 ' 57 ■■ 

Belgium French 5 ' 2|2 • 27-83 9 ■ 23 13-73 



LS; ■ ■ 2 : 13 33-52 • t ; 55 ; 27-85 

France , , ,1 ~n - 1 31 - 



7 72 M 5 ,, 58 ; 3H0 , 

Hungary 1 23 ' 1 ^ 32 H; , 3 . 

Ireland ' 1 '51 - ' . 3 ' 25 25-36 

Japan ■ • '3 83 80-93 ' ; : 9 ,, : 79' 68-96, 

Hew Zeaiar-^ ■• 8 ■ ^7 ■ m\ ■ 12 ,43 .21-84 

Sc. and', 2 ' 7L--..-----5y3 . : - 

• Total Possible , .8 15 ' ■ 



1 



Table 12.5 / * 

SUMMARY BP QUAhlTATjVE JU'OGMENT 0F COUNTRIES 

(Popalation Aj 

. . ; ; Bore Tc^st 

1. . Fractions 



1 .... ■ -. • - ■ . • 1 

WJ ieri Taught Apprbpriatehess 

L% of ItemlL , 

P T S 

Belgium I F!) 50 50 0 ^ 1.5 

Japan 100 0 0 1.1 

US 0 100 0 2.0 

Canada 0 ; 100 0 ; 2.0 

' " ' ' ' '■■ 

^^^stralia 38 *62 0 1.9 

Spain 75 25 6 1 .6 

Chile - 88 12 0 2.0 

X=1 .72 . SD=.35 



2; Percent, Ratio and Proportion 



Belgium (FI) ' 50 — 50 .5 
Japan . _ 100 --^ 0 ^ : , 1. , 0 

Canada , — 100 0 2.0 

Australia 12 8)3* 0 . 1.9 

Spain 100 ~ 0 1.4 

; Chile —MOO 0 " . ' 2,0 

X=1 .54 SD=.59 



3* Algebra 



Belgium (FI) 


=- 100 


0 ■ 


2.0 


Japan 


75 


25 


1.1 


US 


— 100 


0 


1.1 


Candida . 


88 


12 


1.6 


Australia 


— *10b 


0 


2.0 


Spain 


88 


12 


1 .6 


Chile 


12 


88 


2.1) 


<•,. 




X=1.63 


SD=.'40 



For key, see bottom oext page. 
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table 12.5 (Cont'd.) 

U. Gebrrietry ".; 



Bejgjum (Fl) 
Japan 

US 

Canada 



(% of Items) 



12 



T . S 
— 180 
38 5b 
88* *88 
25. .75 



N 



12 



Appropriateness 



b 

1.8 
.6 

i4 



1 



Australia 

Spain 

Chile 



5. Measurement 

Belgium {Fl) 

Japan 

US 

Canada 



12 

37 



100 
87 

13 



62 
62 
62 



13 
100 

87 



25 
0 
37 



X=.81 



1.5 
.6 

1.6 

SD=.58 



.12 
1.3 
2.0 
2.0 



Australia 

Spain 

Chile 



13 87 
100 
25 75 



X=1 .5 



1.7 

-1.6 
2.0 
SD=.68 



Grand Mean =1.3 . 

Grand Standard Deviation = .67 



key 

Wh^n taught 

P Prior to this yea?' 



(that is, taught at a lowar level than 
8th grade but not snecifically taught 
in 8th grade), 
(that is, either ihtroduded in St'^ 
grade for the first time or reviewed 

retaught in 8th grade), 
(that is, riot taught Up to or iriclud- 
irig 8th grade but riot taugh^t iri 9th 
or hJgher grades). 
Not in curriculum (not In cUrricuJUrr?, at any grade level), 



Taught this year 



Subsequerit years 



Apfjropriateriess 

0 inapprOprit::*e 

1 accoptable 

2 highly app'-bpriate 

N*'.-: if the J'erri tests knowledge or ski lis taught ur tv 
'evsl Of the target population and o:- t>« • 

difficult or easy at the end of the 'ear ih'in it <t 
b' ppropriate. 
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Table 12.5 (eoht'idj 

SUMMARY BY GiSyNTRY QF THOSE SUBTESTS 
THAT FIT A eOUNTRY'S f.;!RRieubUM 



T O P ! C 



CbUritfy J't'ivvtibhs Percent, Ratio Algebra Geometry Measurement 
: Prbportibri , 
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TABLE 12;8; BETWEEN SGHiSQL VARIATION (item Prbpdrtlbris S Subtests^ 
CANADA/USA 



N = 15 Schools 



N =381 Students 



FRACTIONS 



Between School 



item 


item Statistics 


= Statistics 












:: Hi 


-SB- 




1. 




A9 


.3k 


1.0 


.25 


.17 


2. 


.72 




.31 


i.d 


.21 


* 1 9 


3. 


.69 


.k6 


.15 


1.0 


.2k 


.21 


. k. 


.72 


• A5 


.31 


1.0 


.18 


.10 


. 5. 


. A3 




.07 


1.0 


.26 


.16 


6. 


.79 




.38 


1.0 


■ .18 


.lU 


^ 7. 


.38 


.k9 


■0 


.8 


.22 


1 rs - 


fotals 


It. 3 


1.9 .-. 1 


.5 


6.8 


l.U 


.3k 


PERGENT, RATIO, PROPORTION 














Betveen 


School 




Item 


Alt eiiL;_Stat ist ic s 


Statistics • 








SD 






-SR- 




1. 


'.3U 


.U7 


0 


1.0 


.29 


.21 


2. 


.60 


.kg 


.31 ■ 


1.0 


.21 


.11 


3. 


.25 


.U3 


0 


.6 


.16 


.07 


k 


rm 


.U9- 


.28 


1.0 


.21 


.08 


ais 


1.3 


i.O 


.83 ' 


3.6 


.8 





Item item Statistics 



Betveen School 

St atis^ic^ — . — ^ 





X 


SD ' 


Lo 


Hi 


SD 


ETA^ 


1. 


.Ik 


. .U3 


.5k 


1.0 


. .16 


.11 • 


2. 


.ky 


, ..U9 


..19 


.9 


.20 


.10 


3. 


.18 


.39 


.02 


.k 


.±k 


.10 


k. 


.62 


.U8 


.22 


.93 


.25 


.23 


5.' 


.18 


■-39 


0 


.6 


-13 


.03 


-als 


2.2 


1.2 


1.1 


3. It 


.69 


.2k 



GEOMETRY 



^ _ _ __ _ Between School 

Item item Statistics Statistics 





















SD ■ 


Lo 


Hi 


SD 


ETA^ 


i. 


.70 


.U6 


' .li5 


i.b 


.18 


.11 


2. 


■.kl 


.U9 




.89 


.22 


.12 


3. 


.25 


.U3 


..15 


-5 


.16 


-Oil 


k. 


.17' 


■ .'38 


d 


.8 


. .21 


.12 


jtals 


1.5 


.97 


1.0 


3.0 


.k9 


.Ik 


AtSESRA 


















Between School 




item 


item 


Statistics 


Statistics 







--X 


SD 


Lo 


Hi 


SD . 


ETA^ 


1. : 


.111 


.U9 


■ 07 


i.e 


.25 


.12 


2. 


.57 




.23 


.9 


.16 


.06 


3. 


.70 . 


.li5 


.38 


i.O 


.21 


.15 


k. 


.k9 


:50 


.08 


1.0 


.26 


.1? 


5. 


.61 


A9 


.08 


1.0 


.26 


.19 


als 


2.7 


1-3 


1.2 


k.9 


.8? 


.32 
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12;3 Proposed structure of the Cognitive Test ,(Popuiation A: 

Lbrigitudirial ) . 

Table 12.9 shows the present structure of the Pop^Jation A testj 

This structure differs from what is given in Bulleti^n 3 (pages 28-29) ^J^^ 
that the present design calls from stratified random assigjiment of jtems 
across the rotation forms where the strata a re subject matter class if i- 
catibhs from table 12.1; The anticipated requirements for tesjmg time 
for the present structure are as reported In Bulletin 3, page 34. 

A proposed revised structure is given in Table 12.10. The hew 
features are; 

►■J - . - . - ■ ■ - 

(I) The required common core for all countries has been reduced 
from 8 Items per scale to ^ items. This set of items, the In- 
ierxtatiorial -£ o re , is comprised of those which are taught'Tb 
the target class during the school year land therefore^ pre- 
sumably, hold promise of reflecting growth}. 

(il) The remaining ^ items in the core are chosen by tJle coun- 
tries to best reflect their curriculum or to otherwise respond 
to natlbnaL interests^ specifically as they relate to classroom 
process. This set of items is the National Core . 

(iii) The present rotated forms (4_ with 32 items each) may be 
augmented by the countries from the international pool of 
items. 

Civ) In order to provide a comprehensive growth measure, all 
items should be administered in the pretest and posttest. 



Topics 

Fractions 

Percent 

AjgelDra 

Geometry 

Measurement 

Subtotals 

Table 12.9 



"Existing" (A/ 124) 

Cora Test Rotated Forms 
# of Items 



8 

8 
8 
8 
8 



Rotated Form 1 
Form 2 
Form 3 
Form 4 



1.- 



4-of-i terns . 
32 
32 
32 
32 



46 



128 

TOTAL ITEMS = 168 



Sequence of Testing: Core - Pre and Post; Rotated Forms - 
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"Proposed" 

Topics Ihterhatiohal Floating Rotated Forms 

Core or . 

Natibhal Core 



Fractions 4 varied bri #of forriib and 

Percent 4 ' cbnteht and length of forms 

Algebra ^ difficulty dejDerids bri cbUn- 

Geometry U dejDehdihg bri try's chbice of 

Measurement • 4 ' cdUhtry choices "sampling" plan 



Table*12,10 Subtotals 20 20 ,120 or more 

Sequence of Testing: International Gore and other items given both Pre 
and Post. 



12.4 Demands of student time vs. options for analysis 

> From the point of view of the researcher, it is desirable to have 
as n^any studen^ts as possible respond to as many items as possible. If 
al 1^ tu dents repond to al [ items / items a re sa id to^ be comp letely crossed 
with students. The completely crossed design per^ flexibij- 
'Ity for analyses:^ ^ between students, between classes, between schoojs. 
However, this option would require an estimated 4 hours for an inter- 
natibn.al pool of 166 items. For the vast majority of countries,, this is 
undoubtedly ah unrealistic expectation. 

. From the point bf view of minimal intrusion into the sehbo! (minimal 
administration time) , the preferred test design Is that of matrix sampl- 
ing, one version of which is rotated forms.^ This is the design pro- 
posed for Pbpulatibh B for twb reasbns: (13 a grbwth measure is not 
required, except as a hatibhaL bptibh (2;^ for many cbiihtries^ testing 
time appears to be more difficult to bbtaih at the advanced grade 
levels. The analyses which such a design permits are those at the 
classroom level. ■ 

Under the rotating form scheme, each item in the pool is taken by 
a random sample of students. For example, if there are foUr rbtated 
forms, each Item is taken by | of any sampled class. ♦ 

The proposed structjUre for Population A is an attempt to acknow- 
ledge the many^ constraints upon the^ instrument as well as the demands 
of the objectives of the Study. 

The remamder of thjs section presents a proposed procedure for 

designjng the cognitive test for the longitudinal study, Popu 

the light of comments from the varjqus countries and the empirical in- 

formatiph on the items which has been received to date. 

The proposals in summary are as follows: 

1. The main cognitive te^t forms should be constructed as stra- 
tified random samples bf items from the whole pool. 
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2. The number of forms taken by a studerit--^^^^ is^ the density 
of item sampiing— should be a natioha! dptioh, 

3. There should be a national core of items taken by all students 
In a given country, but this core will differ over countries. 

^. There should be an jnterhationa! core taken by all students in 
all countries, . 

5. The pre-test instramehtatlori for the lohgitudmaLstudy should 
be a complete copy of the post-test ihstrUmeritatibri^ with all 
items being used both times. 

These proposa[s and the discussion points which foMow them are 
aimed primarily at the design of the lohgitudirial, Pbjiulatic^^ 
the study, although they are perhaps applicable to the other parts as 
well. In a number of places, one is led to the cqnclusion that really 
good design would require j^relimiriary estimates of difficulty Jevels^^^^ 
items and of difficulty differences dver a school year. Ideaily, the 
overall study plan should leave time to obtain such data, analyze the^ 
and feed the results into a final design. The plarL_al so places con- 
siderable responsibilities, on National Centers in terms of coding, ad- 
mihistratibn, reporting of data arid so forth. 

PROPOSALS 

Proposal 1. Stratified random i nstrument s 

It is clear that in most countries, it will hot be practica[ to have 
all sampled students respond to all the eoghitive items. Some kirid of 
matrix sampling is required. It has been suggested that in certain 
parts of the Study the test forms be defined along ebriteht lines, that 
Js, there would be a form for ratio, one for geometry, etc. However, 
for a number of reasons^ stratified random assignment should be used. 
The major substantive cat^ories of the item pool vvduld define the 
strata; Where possible, further blocking woUld be made, on estimated or 
approximate item difficulty. Each lest form would then contain a strati- 
fled random sartipfe of the paol. That is, the forms would be made up 
by stratified randomizatlbrii 

One immediate effect is that the sampling errors of the estimates of 
item difficulty within a cbriterit area become less correjated. The deter- 
mihation of the distributibri of subtest scores is potentially much more 
accurate. , 

Stratified random iristrumeritation is essential for rejatlohal studies. 
No core test can have sufficient content variability to be a realistic 
measure of achjevemerit^. The sutjsam^ples of students taking different 
cohterit-based forms are difficult to combine in a single analysis, and 
the sa/nple size for any brie form would Hkely be too small for relational 
analysis,. Even classroom means might be too m^^^^ because the 

sample of students would be so small in a classroom. 

But stratified random ihstrumehts^btaih a little iriformatioh jbpUt 
each cbriterit category from each student in a classroom, while gettmg 
the same amdUrit of item data per classroom. Total and subpbbl means 
cari be calculated for each student and they will be comparable over 
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formSi Consequently, integrative analyses can be carried but in which 
all student data are used simultaneously and the dependent variable 
|cbghitive_ achievement in the whole pool or a stratum _df the pool) 
, includes forrtl differences just as measureinerit error. Techhical ac- 
curacy can be Improved in such relatfohal analyses by cross-calibrating 
the different fdrms. A simple method is to Use form as a cdritrdllirig 
variable in the analyses. In any case, the quality df the irifdrmatiori 
with stratified rariddrtl iristrUmehtatidri is higher' arid sd we shduld feel 
obliged to accept any 'computational complications. 

: Proposal 2> Flexible arrangements 

The . practicalities of administration vary across countries. In 
some,_ the sjjccess of the s_tudy will^ depend greatjy on keepmg the 
response burden^ ojn indjv[daals to a iriimmam. l^__others^ there would 
^®_J^°_ P/l°^'^'^ J"^*.^?^ ev^ry student answer every'cognitive item, in 
fact, this might be preferred. 

Because of this Variation, the study design should allow flexibility 
in the instrumeritatibri arrarigemehts, perhaps by defining three levels 
of matrix sampling— light, medium and complete response (ho samplirig) . 
The light sanipling is defined by having each, student take orie^ of the 
stratified random forms, the medium by_ twb (or more?) forms, arid the 
complete by each student's, taking all forms. Atteritibri in the admini- 
stration procedures would have tb be gjven tb assuring rbtatiori of the 
forms, arid the data prbcessigri would have tb ariticipate varylrig 
amdUrits df data. Nd difficult problems cari be seeri. 

The samplirig referrees will, hbwever, have tb ^ watch out for re- 
ductidn in. effective sample sizes when each student is prdvidirig more 
infdrmatidn They should riot alldw the riUrtlber df schddls., for example, 
td be much reduced. 

Proposal 3, National cores 

The statistical accuracy of the analytic rersults, both the national 
and subnational summary item and subtest statistics and^the, relational 
coefflc[ents, can be improved by incorporating a core set of Items which 

is given to alf students in addition to the stratified random forms. 

* - ' . - 

This is_ pr^oposed_^partly as^ a techriical device. Under, the item 
sampling plan, any par tic u [a r item will be g] yen to a sample of students 
sprea^ over many classrooms. li_wL'iJl°* be possible to restratify the . 
students iri each classroom, so the sample within classroom for an item 
will be simple raridbm. But if there is a core test giveri tb all students 
in the classrbbm, theri the item statistics can be derived from regres- 
sion for^mulas and ratio estimates which will have much more accuracy 
than cbriveritibrial estimates. The core can also be used to cross- 
calibrate the test arid subtest scales of the stratified raridom forms, ajid 
this; wljl improve the quality of the pooling of the forms fri relational 
- analysis. 

The determlhatibri bf this cbre ahbuld be made partly bri techriieal 
grdUritfs: orie wants a test which will cbrrelate highly with most cori- 
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tent areas which discrHnlnates welf between students, and which does 
no" h^f floor/ceiling problems. it is^nlikely that a single core w^^ 
have appropriate technical qUaJLty 1" all countnes. especially with res- 
pecf t? difficulty level. Consequently a ^^P^'^^ ^';Zffar^ei^^ 
be ^65ted by each country. When the cores are used for statistiea 
aijuftme^t of responses to the main item pool, the lack of m^^^ 
correspondence will not matter. A strat.f.ed set of ^ems ^ould be set 
aside fcr core Use, and in: each country, the local core would be se 
lected on thrbasis of estimated item difficulty, content coverage, • etc . 
Preliminary item statistics will help in choosing the cores. 

Another reason for including natidhal cores is to allow each coun- 
try to obtain extensive data, especially Pje/pos* jlata^ m areas o^^ 
content Which are of more interest locally and which may not be covered 
in sufficient detail ir the item pool and the^Mnternational core. 

Proposal Intern atiohal core . , 

The original concept of a core test was to provide a Way -to do in- 
ternational comparisons. If . some countries are t^lf^ffJi'^l^f.^j^f "^^^ 
at the floor of a core, then the comparisons are Jimited and the core, 
wit! be useless Tor national analyses in the countries at the extremes. 

Nevertheless, a small international core is. desirable It^^w^^^ 
orovide a auick wav^ to check theb statistics on the complete ^ata, it 
would allow^slS^e quTck initial ahalysis, and it would be a ^w^ay to have 
some results which do = hot require too mach explanatjon . Tf| mtern^- 
tTonal lore^ should be considered^ as a separate issUe from the floating. 

national core. • . 

P^osal 5. Pre-te'st/ pos t-test sc hemes 

In the IbhgLitudinai study, it is strongly ad visaed that the cognitive 
instrumentation for the pre-test ba a complete copy of the post-test in- 
strumentation. That is, the same items should be used.,.. 

One reason is to be able to describe cognitive growth in^ detail , 
that is for each item and subtest. A principal goal of the study is to 
find out how many items, which items, and what kind of items a student 
£^?hrover the^school year. A core test if defined internationally 
would be inappropriate for, each nation separately. 

A second purpose of the pre-test ^^"^ f ,^1^^°"^^^^ 

for assessing the effects of classroom processes. But in this case, tne 
test had better measure student khowledae; relative ^o what is taught 
during the year. Again, an internationaf core wilUbe insufficient for 

this purpose. / . 

GENERAL DlSCUSSmH JOI NTS 

A. Item and subtest ahalysis - ' 

: The primary analytic outputs of the Study will be es^timates of the 
difficulties of the cbgriitive items (each of „the items) and estimates of 
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the distribUtibhs of subtests frdrri the iterti jdooI. The estimates are 
required at the hatibhal level for purposes of ihterhatibhai cbmparisbhSi 
For withih-hatibh analyses, it will be necessary to provide breakdbwhs 
of item difficulty and subtest performance according to educotibhal or 
demographic stratificatipris of local interest. In the Ibngftudirial arialy- 
sis for Population A, the item statistics and subtest summaries heed tb 
be calculatecJ at the begihhihg and end bf the schbbi year, presumably 
with equivalent detail (see Proposal 5), tb evaluate growth. 

While these item and subtest analyses may seem eiemeritary> a 
great deal of statistical arid cdmputatibrial labbUr will be devoted tb 
them. Regression estimates based on core tests can greatly improve the 
quality of the results, arid jackkriife estimates of staridard errors will 
probably be necessary. It wouid be yaluable to ensure that each 
subtest had buiit-in replicatibri. The desigri cdrisideratidris for sampling 
item^s and students shouid^ into account the iikeiy withm-country 

breakdowns as well as whole-country statistics. 

B. Relationa l ana l ysis 

A major intent of the [qngitudinal study is to relate student back- 
ground and classroom process with cognitive achievement, inevitably, 
the crbss-sectiohal studies will also attempt relational analyses. At the 
leasj, there wiji be item and spbtesl breakdowns by type of student 
(e.g., sex) and type of school or classroom (e.g., track). 

It is importarit tb keep in mind that the analytic goal of relational 
analysis is to estimate ^'elatibrial parameters, such as the regression 
coefficient relating mathematics achjeverrierit to student sex or mean 
classroom achievernent tb bpp)brtdhity to learn. Theoretically, the 
cbgriitive measure for the individual or the classroom should be defined 
iri terms of the whole pool of items or of some complete sUbpRDbl. The 
analysis has to work from the iricbmplete resjDbrise data> based bri jbirit 
Item arid student sampjirig, tb estimate the relatibriships ambrig the 
theoretical . variables. While this certainly will iriydlve calculatirig aver- 
ages arid partial scdres> the accuracy arid sigriificarice bf those iriter- 
mediate calculatidris are dniy critical iri hdw they, affect the fjrial re- 
sults. For example, one is riot fundamentaliy interested in the cog- 
nitive test score mean for a classroom, one wants iristead td kriow the 
regress[on of true mean o If .the items and 

students are too sparsely sampled, there wiii be jtechnicaj difficulties, in 
estimatjiryg the regression. If^ the item pool is substantively restricted — 
for example, by taking a limited core — then the regression analysis is 
inherently doomed. 

G> Griteria for iristrurnehtatibci 

The decisions about instrument arrarigerrierit should be techhically 
based arid given as much hard analysis as are the decisions about 
studisrit sampling. What can we say br guess abbtit the ihter-cburitry 
or- iriter-cbriterit variability iri Item difficulty? Optimization of the 
instrument arrahgerriehts ihvblves cbrisideratibri of data cbsts, student 
time^ arid admiriistrative costs relative td arialytic accuracy. 
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Several notions of accciracy are relevant here: (1) the cdmplete- 
riess of the sampljng of the item pool or subpooW C2) thie accuracy oT 
that sampling, and whether the accuracy can be estimated and adjusted 
for; (?) the accuracy of the student and school samples for items and 
test forms; and (4) the efficiency of the statistical Analysis. 

b. Meth odoiogid a ! pr i nc i ples 

The problem of instrumerit arrangement is cdrihected to the problem 
of student sampling, and a general principle is to spread the content as 
wide as possible. That is, each item or subtest should be administered 
to as broad as possible a sample of students. The matrix sampling is a 
great helpi 

A corollary is that the instrument arrangements should^be designed 
to facilitate pooling of response data over forms or item samples. Rela- 
tional studies should have as much cdhtent variability as^^^^^^ 
eluded in their dependent measures, and this means, in^ particular, that 
analysis based just bri core tests will be deficient (see Proposal 1)- 

12.5 Admiriistratidh df the Cognitive Test 

The IMC alsd prdposes options for test administration. WhicK 
dptjohs a cduntry chooses will depend^ of coarse, bh the amount^ of 
testing time availale and the number of items that are to be admin- 
istered. 

Minimufn Expectation: At both pre- and post-test times a country 
administers the International Core (20 Items) and the rotated forms (120 
items). 

a) With heavy sampling— all students take alj items that would 
mean each student would take 140 Items pr about 3 hdUrs df testing 

time. _ _ _ 

b) With light samplihg--studehts take the International Core plus 

one rotated form — each student would take 50 items at both pre- and 
post-test time. 

Example of one option: At bdth pre- and post-test times a country 
chooses to administer the jriterhatidnal Core plus a National Core plus 
the rotated forms plus additidnal items. _^ 

al_ With heavy sampling this could mean that students would take 
up to 250 items at both the pre- and post-test sessons. 

bj__ With light sampling a student could taken the Iriterhatibhal 
Core J^O)^ the national core (2) ,_ a rotated form (50) and Effectively 
take 80 items at both pre- and post-test times. (This assumes four 
rotated forms df 50 Items each, 20 of which are chosen specifically by 
the cduntry. ) 

12.6 Procedural Steps to Insure the Quality of the Cognitive Instru- 
ments 

To be able to provide a valid item pool Upbri which cduntries can 
make vajid choices it is necessary to gather additjbhal evidence abdut 
the suitability of the cognitive Items. What is needed is bdth universal 



108 



9S, 



cjuahtjtative ihformatibri {item difficulties, estimates of amoant ^f 

growth) and more qualitative iriformatioh [appropriateness of item for 

growth^ arid when and to whom the content arid behavior apori which 
the item is based is taught. 

In order to get this iriformatibri from countries, the IMC proposes 
the fdlldwirig set of procedures. 

For each participktirig cdUritry 

1. Selected items in the Iriternatiorial Item Pool should be re- 

administered to two groups from students: those in the grade that con- 
tain the target population plus the appropriate grade. Since these data 
must be cqllected prior to January U 1980^ this implies that 

a^ Northern Hemisphere countries should test students in 

the target grade and the next higher grade; 

b. Southern Herrnsphere ^ c^ the target 

grade arid the grade level immediately prior to the target grade. 

. Jt is suggested that a minimum of 1 00 students at each grade level 
be administered each item;^that the sampje be a judgment sample that is 
represeritative of the pqpulaJ:ion^ and that there be documentation of the 
types of class arid school in which the testing took place. The empir- 
ical results of this trial will provide the following iriformatiori: 

1. item difficulties for all items iri the pool 

2. an empirical estimate of the growth FX)teritlal of the Item 

3. a rriearis to estimate the betweeri class arid between school 
variability iri item ressfX)rises. 

In addition to this qtiarititative informatibri the IMG would like qual- 
itative informatibri about each item at the teacher level . This IricluSes 
when the item is taught> to what proportion ■ ot the target population, 
and subjective judgments of the pbteritiar grbwth. 

The IMC will organize the quantitative arid qualitative informatibri 
and provide it to countries as a basis upon which, to "tailbr" the cogni- 
tive instrument in line with the suggestions cbritamed abbve. 

The IMC recognizes the increased burden that countries are beirig 

asked to assume. _We believe, however, the need to be responsive tb 
natibrial concerns and the complexity of deyelopihg a sdUrid cdgriitive 
test that will produce vaMd l^nternatlonal results dictate an increased 
kriowledge of the properties of items in^t^^^ Pdol. We ap- 

preciate your cooperation In the past and look forward to your corn- 
merits ori this profDosaf. 
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As stated In Bulletin No. 3^ the ihternatiohal repdrtihg of results 
wiii be through a series of voiames. Qne volume will focus ;c)n a ^ur- 
ricuium Analysis; one a comprehensive but not overly technical report 
of ail the major findings of the study; one a technical volume that 
shows how a variety of statistical models of a highly technical nature 
can be used to analyze the data and; finally, a series of communications 
that highlights the most pertinent fihdih/gs of the study, written in 
popular language and addressed to lay audierice. 

The schedule of cdmpletibri for the Shternatidna! reporting follows^ 
roughly, the timetable for the lohgitUdirial study. Jhat is to say, data 
are to be collected by mid-1982, analyzed by mid-1983 reports are to be 
completed by mid-1984. 

Each of these reports will utiMze.the results of thje study in dif- 
ferent waysi /Each requires arialyiis arid presentation of dat^ 
differerit levels of techriical arid statistical sophistication. These re- 
quirements combined with the cbmplexity of the desigri_ that is envis- 
aged, make, as iri previous studies, the calculation of statrstlcs and 
dissemiriatibri bf results a fbrmidabie task. Not only is it necessary to 
know how to calculate the appropriate statistics but also results must be 
provided quickly arid efficieritiy to those who are responsible for re- 
jDbrtirig the results. 

Iri addition to providing results to _authors of various volumes, 
plus complete data tapes to authors of the technical volume, fDresent 
plans call for the dissemination of procedures and software to facilitate 
within cdUritry analyses. The latter places additional impbrtarice bri 
proiDer^ data ariaiysis and appropriate statistical procedures. AlthbUgh 
it will be compjex technicaliy to produce some of the desired results, it 
is essential : that simple procedures and adequate sbftware be provided 
so that countries can produce both analyses cbmjDarable to those dbrie 
internationally and ones that are uniquely appropriate tb them. 



m. statistics and data analysis 

Tables m. 1 through ^^.7 dutime an Initial attempt to d^^ 
variables in the study and to indicate the kinds of statistics that will 
be computed. JUst as other tables tend to make the complex appear 
simple, so too for these. Given the design of the study some of these 
statistics are extremely difficult to compute. There are issues of the 
rotated forms, sampling decisions/ weighting, levels of analys^ls, esti- 
mating standard errors, appropriate regression equations, and many 
others, embedded in the problems of what startles should be computed 
and how they should be computed. 

Below are some guidelines that have b^^ discussed in relation to 
data analysis and the calculation _of^ appropriate quantitative results. 
This li^st is not aM jncjusjye and some guidelines have benefitted from 
more discussipn than others. A thorough discussion of these issues 
and responses from the countries to the Interhatioh^ Cbm^ 
mittee (IMC) will, we hope, add to and clarify these general guidelinesi 

» - - - -- — 

ia.1. Measures of Central Tendency and Variability 

The tables contain marks where measures of central teridericy arid 
variability are to be cbmputed. It should go without saying that such 
measures will be chosen that best reflect the type of variable being 
analyzed and the kind of information to be gerierated. 

14.2. Item Analysis 

Though riot specifically stated iri the tables, both the cognitive 
tessts arid the attitude scales will be arialyzed for their psychomMTJc 
properties. Such analyses include item discrimination parameters, 
factor arialysis (of various sorts), latent trait analysis and other appro- 
priate statistical descriptdrs and methods.'' 

l^t.B. Levels of Analysis 

The fulT study Ts 'tesed on a sampMng p|ari that iriclades^ studerits 
within classrooms, c[assroqms wthm schools and schools within coun- 
trjes. AJthq^ugh ihe question one asks should always precede the statisr 
tics that are computed, it is important to keep in mirid the different 
TeytlfT^f ~^rialyses that are possible. Of particular importance iri this 
regard, are the distributions of variables within classes. Preserit plaris 
call for the calculation of a riumber of descrifDtbrs of these withiri class- 
room distributiorisi 

m.4. Staridard errors. i 

It has been suggested that as a gerierai pririclple rib statistic be 
dissemiriated without ari hbriest staridard error attached to it. This 
implies the jackknifirig bf all estimators iri order to get appropriate 
standard errbrs. 
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lUiS. Weighting 

It is unlikely that samples wlM be self-weigh ting. This implies the 
necessjty of using weishting schemes in the calculation of statistics. 
Such is especially the case where countries might choose to over sample 
a particular stratum in order to address an important question. On the 
other hand there are some analyses where weighting might not be 
important (e.g., between country analyses) . In general the^uideline 
is to weight appropriately where necessary and to present both 
weighted rnd unweighted results where they will provide more insightful 
interpretation of results. 

14.6. Corrected Scores 

Given that there are many correctidns for guessing and none 
available given the design that will provide an apprppriate correctio^^ 
in general the results should be produced without corrections for gues- 
sing, those Countries or individuals who wish, to correct would be free 
to use the correction they deem most appropriate. 

14.7. Standard Scores 

When preseritirig profiles of various scores and results it may be 
desirable to express scores in standard units. It is suggested that 
those standard units be chosen so as to eliminate negativ^ _^and non- 
integral scores. For example, a within country metric might have a 
man of 50 arid staridard. deviation of 16; a between country metric 
might have a meari of ],00 arid standard deviation of 15. 

What follows are tables that beg iri to delirieate what kjri^ds of statis- 
tics will be computed for the vanables that ^re to be iri the study^. As 
the study becomes more refined, the variables may change and how 
they will be handled statistically could charige also. 

Orie assumption upon which the tables are based is that countries 
will use rotated forms to gather info rrriatiari about students' cog mtiye 
achievements. A second is that the sampling ^plari will be based on 
samples of classrooms within school, thereby making the level of analy- 
sis issue extremely i^mportant. As a corisequerice, many satistics must 
be computed for students withiri classrobms> classrooms within schools, 
and schools. 

' Although, giveri the desigri of the study, some of ihese statistics 
will be difficult techriically to compute, the general goal is to produce 
estimates that are relative, common and straightforward. 
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* STATISTleS AND DATA ANALYSIS 
U.l. SCHOOL QUESTIONNAIRES (P0PUhATl0NS A AND Bj 

VARIABLE STATISTICS _ 
o Measures of 

Freqaericy Central 

Distribution Tendency Variation 

1; Community Served X 

2. Enrollment / X X X 

3. Enrollment: Sampled Population X X X 

4. Enroiiment: Sampled Populatidri 

Mathematics X X X 

5. Number of Teachers X . 

6. Number of Mathematics 

Teachers ^ X 

7. Mathematics Prefej red 

teaching subject (teachers) X 

8. Number of school days /year X X X 

9. Number af peribds/day X X X 
10. Length of periods X X X 
Jl. Calculators: Encouraged ^ X 

12. Staff meetings: teachers 

' 1 mathematics X 

13. Staff meetings: activities X 

14. Calculators: Policy, 

Four functix)hs X ^ 

15. Calculators: Policy, 

Programmable X 

Pi^oupingi Policy X 

17. Gender: Policy X 

Central Tendency: p = proportion,^ x = mean. 

Variation: SE = standard error, SD_= standard deviation, SS = both. 
Level: 1 = student, 2 = classroom, 3 = school, 4 = all levels. 



Level 

3 
3 
3 

3 
3 



3 

3 
3 
3 
3 

3 
3 



3 
3 
3 
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Vn.2. TEACHER QUEStidNNAlRES (P0PU tATIiSNS A AND BJ 



VARIABLE 



Frequency 
Distribution 

Gender X 

Age . X 

Teaching Experience - 

Total X 

Teaching Experience - 

Mathematics to Pop; X 
Preparation - Mathematics , X 
Preparatj'bri Pedagogy : 

Mathematles X 
Preparatibh - Pedagogy: 

General X 
Teaching Hours /Week: 
^ Total ; ■ X 
Teach ihg Hours /Week: 

MathernatLcs X 

Additional Duties X 
Teaching Schedule . • 

a} Pop. A or B - classes; hours X 

b] Lower - classes; hours X 

ci Higher - classes; hours X 

Target Class: Subjects Taught X 
Target Class: Number of Teachers X 
TaVget Class: Number of Students X 
Target Class: Periods 

<7 Ihstructlpn/Weej< X 

Target class: Length of Period X 
Target Class: Hours 

Instructjon/Yeaj" X 

Target Class: Compared to 

others in School X 

Target Class: Ability Range X 

Target Clais: [hitial Mastery X 
Target Class: Subjec t Matter - X 



STATISTICS 

Measures of . 
Central 
Tendency Variation 



1. 
2. 
3. 

H. 

5. 
6. 

7. 

8. 

9. 

19. 
11, 



12. 
13. 
19. 
15. 



16. 
17. 

18, 

19. 
20. 
21. 
22. 

23. 



25. 
26. 



Target Class: Compared to 

Country pbpulatibh 
Target Class: Activities Teacher 
a) Last Week 
bj Typical Week 

Target Class: Activities Student. 

a) Last Week . 

b) Typical Week 
Target Class: Students 

Respbrid tb QUestidhs 
target Class: Varying 
Assign.mehts 



X 
X 



X 
X 



X 
X 

X 

X 

X 
X 
X 
X 



X 
X 
X 



X 



X 

X 

X 
X 

X 

X 

X 

X 



X 
X 
X 



X 
X 



Level 

2.3 
2,3 

2,3 

2,3 
2,3 

2; 3 

2,3 

2.3 

2,3 



2,3 
2.3 
2,3 



2,3 

2,3 
2,3 

2,3 

2.3 
2,3 
2.3 



2,3 

2,3 
2,3 

2,3: 



2,3 
2,3 
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TEAGHER QUESTIONNAIRES (CONTINUED) 



27. Target Class: Hoars 

HbrneWdrk Assigned 
a) Last Week 
L_ bj Typical Week 

28. Target Class: Calculators - 

Access 

29. Target Class: Calculators - 

Use 

30. Target Classj Topics Covered 

a) Number of weeks 

b) Spiral 

31. Target Cjass: Materials 
^32. Target Class: Textbook ; 



X 


X 


X 


2.3 


X 


X 


X 


2,3 


x^ 


x^ 


x^ 


2,3 


x^ 


x^' 


x^ 


2,3 


X 


X : 


X 


. 2>3 


X 
X 
X 






2,3 
2,3 - 
2,3 



^This question yields at least two variables; frequency of use with what type of cajculato 

2 - 

This question olso yields at least two variables; how used with what type of calculator 



Possible Composites: 

A. Allocated Time: Tot§i-Var 15 x Var 16 (Var 17 provides a check) 

B. Alfocated Time: Total Instructionai - Var 15 x Var 16 x ((Var 23b '+ 
Var 23c) /Var 23a + . . . + Var 23f) 

C. Allocated Time: Instructional r Common Fractions Var 15 x Var 16 x 
{(Var 23b + Var 23 cj/Var 23a + . . . + Var 23f) x (Var 30a) 

D. . Allocated Time: Instructional - Decimal Fractions (as above except 

substitute Var 30b for Var 30a) 

E. Allocated Time: Instructional - Rat[q ^nd Proportion (as above except) 

F. Allocated Time: Jhstractlonal - Percentage (as above) 
C. Allocated Time: Iristruetibhal - Measurement (as above) 

H. AMdcated Time: Ihstructibhal - Geometry (as above) 

I. Allocated Time: * Iristructjorial - Formulae S Equations (as above) 
J. Allocated_Time: Ihstructibhal - Directed Nurnbers (as above) 

K. StUderit Time - Var ma + Var 24b + Var 24c + Var 24d 

L. Materials: Variety - Var31a + . . + Var 31g 

M. Materials: Individualized - Var 31c + Var Bit + Var ;3lg 



Possible Clustering: Variables: 



One teacher - Var 11 combined with Var 12 and Var 13 
Ability - Var 18 combihed with Var 19, Var 20, Var 21 and 

leacheF; Mathematics Preparation: Var 5/(Var 6 + Var 7) 
Teacher: IhdlviduaMzatlqn: Var 24d/(Var 24a + Var 24b + Var 24c) com- 
bined with Var 25 and Var 26 . 



A; Target Class: 
B. Target Class: 
Var 22 
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14..3;'^tODENT aUESTIONNAIRES (POPULATIONS A AND B3 

VARIABLE . ■'■ ; STAtpieS : • 

Measures of 

Freqtiehcy Central ^ - ■ - 

blstribation Tendency Variation Level 

1- Gender - x « 

2. Age ■ : g ' ^ y « 

3. Father's Qecufjatibri (Scaled) X y ^ U 
i». Mother's OccUfjatibh (Scaled) X a 

5; Father's Education tScaled) X X "5 « 

6. MothecJs Educatibh (Scaled) X 'v a 

7. LahguagelSpoKeh in Home X 

8. Planned Further Education ^ X 

9. Rjaurs Homework - Mathematics 



n 



X X « 



al Last Week ^ v * n 

b) Typical Week X A % n 

10. HdUrs Homework - All Subjects X X 

11. Hours Outside Tutoring - 

Mathematics ^ y « 

a) Last Week 5 5 y « 

b) Typical Week , X ^ a h 

12. Parents Help with 

Mathematics: Frequency X 

13. Computational Aids: 1 v>1 v"" 



X X n 



What Used 



X' X' 



14. Computational Aids: 2 w2 ii 

How bied X X X M 



15. Home Support for Mathematics 
(18 items) 



X X 



^ This question yields at least two variables; not only what compUtatibhal aids are used 
but also where they are used. 

^This question also yields at least two variables; how computatibhal aids are used and 
which bries fbr what purposes. 



Possible Cbmpdsites: 

A. Student Background - Var 1 + Var 2 + Var 3 + Var 4 (Weights for composite 
may be chosen in a hUmber bf ways) - - - . 

B. E>iposure to Mathematics Outside bf School - Var 9 + Var 11 (Composite to be 
formed of either the "a's" br "b's") '. ,^ , . u , 

C. Environmental Support for Mathematics - Var 12 + Var 15 (Scale to be formed; 
potentially tliece are 14 items) u i ^ i 

: b: Computational Aids: Access - (Var 13a + . . . + Var 13ej 
E. Computational Aids: Use -- (Var 14a + . . . + Var 14 e) i 



lie 
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lit. 4. COGNITIVE TESTS (POPULATiON A) 



' VARIABLE 



Frequency 
bistributidn 



STATISTICS 

Central 
Tendency 



For Pre and Posttest or Posttest only 



U f 



Variation 



Level 



1. item responses (about 160) 


X 


p 


■ SE 


1,2 


2i Subtest: Fractions 


X 


X 


ss 


4 


3. Subtest: Percent^ Ratio^ 










Proportion : 


X 


X 


ss 


■ U 


4. Subtest: Algebra 


X 


X 


ss 


I* 


5. Subtest: Geometry 


X 


X 


s s 


« 


6. Subtest: Measurement 


X 


X 


ss 


4 


?• Other Content Subtests as Speci- 










1: fied (a,_g./ whola numbers] 


V 
/\ 


V 


ss 


4 


8. Subtest: CompUtatibri 


X 


X 


ss 


4 


9. subtest: Cbmprehensfbn 


X 


X 


ss 


0 


10. Subtest: Higher L^vel; Behaviors 


X 


X 


ss 


4 


11. Other behayidral level subsets 










(e.g.> minimal competence) 


X 


X 


ss 


4 


Growth Scores 










1. Item responses (Raw Gains) 


X 




SE 


1,2 


2. Subtest: Fractions (Raw Gains) 


X 


x 


ss 


4 


3. Subtest: Percent^ Ratio/ 










Proportion 


X 


X 


ss 


4 


M. Subtest: Algebra 


X - 


. X 


ss 


4 


5. Subtest: Geometry 


X 


X 


ss 


4 


6^ Subtest: Measurement 


X 


X 


ss 


4 


: 7. Subtest: Other content 


X 


X ' 


ss 


4 


8; Subtest: Computatfoh 


X 


X 




4 


9. Subtest: ^Comprehension 


X 


X 


ss 


4 


.10. Subtest: Other behavioral 


X 


X 


ss 


4 
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COGNITIVE TESTS (POP^ULATjON A) CbNTINU.'b 



VARIABLE 



Frequency 
bistribution 



Profiles within arid between cduritry 
• (pre and pdsttests or posttests only) 

1. Item reporises 

2. Content Subtests 

3. Behavioral Subtests 

items aggregated according to issue 
(e.g.* roie of applications) : 



Central 
Tendency 



X 



Variation 



SS 
SS 
SS 

SS 



i-evei 



1.2 
1,2 
1.2 

1.2 



Profiles w|thin and between country 
(growth) 

1. Item responses (Raw Gains) 

2. Content subtests 

3. Behavioral subtests 
U. Other subtests 



P 
X 

X 
X 



SE 
SS 
SS 
SS 



1.2 
1.2 
1.2 
1.2 



Anchor Items 

1. Dif^rerices at item level 
■2. Differences at subtest level 

3. Profiles within and between 

countries 

4. Growth between pre and ^ 

pdsttests on. anchors 



X 
X 

X 

X 



P 
X 

X 
X 



SE 
SS 

SS 
SS 



1 
1 

1 

k 



Variance Components Analysjs (Random 
Effects Analysis of Variance) 

1. Items by Students 

2. Subtests b y Students 



Iricluding Fractions , Ratio, 
proportion and percent; Algebra, 
Geometry, Measurement arid other 
coriterit areas. Also, possibility of 
dbirig it for behavioral levels. 



To estimate componerits of variarice due to ^ 
studerits, classrooms, schools arid iriteractibris. 



■ Us 
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eOGNITIVE TESTS pePUbATidN B 
The statistics to be computed are similar, to those for Population 

?> 

a J The»^e Will be ho growth scores : * 

b) The topic subscores will Include: 

1) Anthmetic/humber systems 

2) Algebra/ polyriomihals; equations and inequations 
3} Cebmetry/trigohbmetry _ ' 

4) Analysis/ functions; differeritiatibri; ihtegratibhs 

c) Subscbres within behavibral leVels are also envisaged- 

d) item responses will -be tabulated 

e) Within and between' country profiles will be generated 

f) Analysis based on anchor items will be conducted 
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.14.5. ATTITUDE MEASURES (BOTH POPULATiONS AND TEACHERS) 
VARiABhE 



Frequency 
Distribution 



X 
X 
X 
X 
X 
X 



1 ; Mathematics In Schdb! 
a) item responses 
bj Subscale: Like tiaiphaj 
e) Subscale: LrnpKSrtant 

d) Subscale: Difficult 

e) ' Whole Scale 
fj Subscale: Growth 

2. Mathernatics as a Process 

aj Lterri responses X 
bJ FUN Scale \ , X 

3. Mathernatics and Utility 

a) item responses X 

b) , Full Scale (alpl;ia) ^ X 

^. Mathematics and Myself 

a) Item responses • X 

b) ' Whole Scale (alpha) X 

5. Mathematics Anxiety 

a) item responses ° X 

b) Whole Scale (alpha) X 

6. eaicSlators and GbrripUters 

a) rterri Responses , X 

b) Whole Scale (alpha) • X 

7. Within and Between Country Profiles 
a) All attitude scales 

arid sUbscales X 



Ceritral 
Teridericy 



X 
X 
X 
X 
X 



X 



X - 



Variation 



SS 
SS 
SS 
SS 
SS 



SS 



SS 



SS 



SS 



SS 



SS 



Le^el 



i;2 

it 
4 
4 
4 



1.2 

4 



n2 



1,2 

4 



1,2 

4 . 



1,2 

4 



1,3 
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I'i.e. CLASSROOM PROCESSES AND OPPORtONlTY TO hEARN tpePUbATleN A) 



;VARIABLE 



2. 



Opportunity to tearn 
a) 



Teacher's responses 

1) Item responses ^ 

2) Subscore: Fractions 

3) Subscore: Ratio, 

proportibh, pereerit 

4) Subscore: Algebra 

5) Subscore : Geometry 

6) Subscore: Measuremeht 



Frequency^ 
Distribution 



X 
X 

X 
X 
X 
X 



cj 



Stuideht's responses 

1 ) Ltem respbrises 

.2} Subscore: All above 



Within and Between 
Country Profiles 



Classroom Processes (variabies 
in the process of being refined) 

a) Item responses (e.g^ , 

number of interpretations) 

Fractions 

Ratio, Proportion, Percent 

Algebra 

Geometry 

Measurement 

b) Composites (e.g., degree 
of ihdividualizatibh , 
teaching strategy) 

Fractions 

Ratio, Proportibh, Percent 

Algebra 

Geometry 

Measuremeht 

cJ Withih ahd Betweeh 
Country Profiles 



X 

X 



X 



STATISTICS 

Centra] ^ 
Tendency Varlatlbh 



X 
X 
X 
X 



X 
X 



SS 

ss 

SS 

ss 
ss 



ss 
ss 



ss 



ss 



bevel 



2 

2 



2 
2 

2 
2 



1,2 
1,2 



ss 



t21 
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14.7. BIVARIATE STATISTICS 

1i "Grand" ebrrelatibh matrix . 

a) At stadeht, class, arid school levels 

b) Ambrig all cbhtlhUdUs variables 

cj Including: _ 

1) Subtests of the Cognitive Examination 

2) Altitude Scales 

3 ) Teacher C erieral cha racteri sties ( I n cjudi ng com pds i tes ) 
n] Home background variables (Inc|u^dln^g_compositesj 

5) Home environmentai variables (J n^cl^^^ 

6) Topic specific teaching practjces (Including composites) 

7) Topic specific Opportunity to Learn 

8) School variables 



2 . Other apjDroprJate " reja t lona[ a nal yses " 

a) At student, class ar^d school^ levels : 

bj Among categqrlcat and continuous variables 

c) Calculation of "statistics" such as 

1) Eta 2 ' 

2) Omega 

3) Cohtlhgehcy Table cbefficiehts 

4) bog linear models 

3. One type of Multivariate Analysis (Growth) 
a) At the tbpic level 

1) Students PbbLed Within Classrooms: 

Topic Specific Pbsttest regressed on Tg^ specific 
pretest. Background variables. Opportunity to hearh. 
Student level instructional variables 

2) Between ciassrbdm analysis ^ 

Aggregated Posttest regressed on (Predicated posttest 
scores from pooled within classroom regression, classroom 
variables, sch 
bj Technical recommendations 

1) Use dummy variable for rotated form 
* 2) Use unstandardj^ed coefficients 

3) Jackknife to get standard errors 

k) Should be properly weighted (depends bri sampling plan, 
point of view about weighting) 

5j Should make assumptibh checks (Hbmbgeheity bf com- 
posite regression (from 1); robustness bf regression) 
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15. TIMETABLE OF KEY DATES 

The fblldwihg timetable is reprdduced for convenience from Buiietin 
3 and ajDpIies to Population B. For Population A, (longltudlnai) - 
cbUritries shbUld add one year td_ the key dates for sampling plans, 
translation, administratidri, and so forth. 

To ensure inclusion of results in the international report (see 
Sectidh 13h cdUntries are urged at this time to plan to have completed 
data collection by mid-^1982. 
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TIMETABLE FOR SECOND lEA MATHEMATICS 

Start 



Curricuium Analysis 

Preliminary analysis 

NatlDnai responses to 
international grid 

IMC Meeting 



i^lanr^ing of earricalam 
Analysis Model 



Preparations for Garrlcalam 
Analysis Sympbsium-Gommittee 
to write ap naiional statements 
from Working Papers I and VI > 
textbooks, examihatibhs, etc. 

National Centers to identify key 
hatibhal matheiiiatics experts 

Curriculum Analysis Symposium 

Ihternatiohal Report: data 
analysis^ editing of SymjDbsium 
proceedings and papers 

Publication of Volume I, 
Curriculum Analysis Report 

Cognitive Test Construction 

Identify International Test Grid 

Natk)nal responses to Inter- 
national Grid 

Field Trial of item collections 

IMG arid National Centers write 
new items to fill gaps in grid 

Review trial data and extra_iiems 



October 1976 



January 1977 



August 1978 

January_1_979 
September 1979 
.February 1980 



May 1978 



September 1978 
September 1978 



September 1979 
January 1980 



September 1977 
May 1978 

May 1978 - 

May 1978 



STUDY 

. Cbitiplete 

January 1977 
August 1978 



September 1978 
February 1979 
September 1979 
March 1980 



August 1978 



December 1978 

January 1979 
August 1979 

January 1980 
July 1980 



September 1977 



July, 15, 1978 
January 1979 



August 1978 

.Aagas_t:d:9l8- 
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Ill 



Sta rt Complete . 

IM(j report on trial data and final 

chance for National Centers to : 

contribute items for trial September 1978 September 1978 

Response to IMC report September 1978 December 1978 

Additional 'field trials as necessary October 1978 December 1978 

Review of field trial data and 
synthesis of item pool by New 

Zealand Coordinating Unit January 1979 January 1979 

Final draft of cognitive 

instruments by IMC February 1979 February 1979 

lnij[al review and comment on 
draft of cognitive instruments 

by National Centers March 1979 March 1979 

PrefDaratioh of manuals September 1978 February 1979 

Dry Run all instruments • 
(includes translation and 

refereejng of national options) AjDril 1979 July 1979 

Final iristrumehts October 1979 

P'rihting arid distributidh of 

iristrumehts October 1979 December 1979 

Administration of pretest ^ 

Southern Hemisphere Feburary 1980 April 1980 

Northern Hemisphere September 1980 October 1980 

Administration of posttest 

Southern Hemisphere October 1980 April 1981 

. Northern Hfemisphere April 1981 July 1981 

V 

Classroom Pro c es s es I nstrument s 

Initial development of^ combined 
qpportunity-to-^learn^ and class- : 
room processes instrument June 1977 August 1978 

New Zealand pilot trial of growth ; • 

scores and classroom instrument March 1978 December 1978 

Cohsul tatiQhs on instrtrme nt - M a y I^R August 1978 

Draft of ihstrumeht August 1978 
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LiifTiited natidhal . trials 

Data analysis of trials plus 
New Zealand data 



Inter national trial as part 
of Dry Run 

Finalize instrument 



Translation, refereeihg, and 
printing 

International Mathematics 
Committee meeting? 



Manuals 

Administration of classroom 
Iristrumeht 

Southern Hemisphere 
Northern Hemisphere 

Attitude Scales 

Rationale > and Jdentiflcat[on and 
cteveiopment of affectives scales 

Piiot trial affective scales in USA 



Start 
August 1978 

January 1979 

April 1979 
July 1979 

bctdber 1979 



International trials of affective 
scales - 

Data analysis of trials 

jM6 report bh field trials 

Review of National Center 
comments 

Additional field trials (if 
hecessary) 

Final draft of affective scales 
Trans latibh^ refereeing of problems 
Dry RUH: 



AprW 1979 



February 1980 
September 1980 



January 1977 
JUhe 1977 

March 1978 
June 1978 
September 1978 



October 1978 
October 1978 

February 1979 
April 1979 



Complete 
becember 1978 

February 1979 

July 1979 
October 1979 

January 1980 

August 1978 
February -1379 
October 1979 

Qctober 1979 

January 1981 
July 1981 



June 1977 
October 1977 

June 1978 
August 1978 
SefDterriber 1978 

Getbber 1978 

December 1978 
February 1979 
March 1979 
July 1979 • 
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Start Complete 



Final ihstrumehts October 1979 

1L_._ _ _ __'_o_I 

Cbmpletidn of manuals - ^ October 1979 

Printing arid distributidri of . 

IhstrUmerits October 1979 December 1979 

Admiriistratidri of pretest 

Southern Hemisphere February 1980 April 1980 

Northern Hemisphere September 1980 October 1980 

Administration of poSttest 

Southern Hemisphere October 1980 January 1981 

Northern Hemisphere April 1981 July 1981 

Student, Teacher, and Schoo! Qu e stionn a ire s 

Draft questionnaire items for 
student, teacher and school 

questionnaires January 1977 July 1978 

• __\ . __ 

Draft questionnaires August 1978 



Iriternatlonai Trial iri cbrijurietiori , . 

with trial of additional cognitive ; 

items September 1978 December 1978 



Arialysis of firial data Jariuary 1979 February 1979 



IHC settle firial draft 

iristrumerits ^ February 1979 



IMC report data to riatiorial ____ 
c^riters February 1979 March 1979 

Questionnaires finalized October 1979 



'Completion of manuals February 1979 October 1979 



Translation and refereeing November 1979 becember 1979 



Administration of questionnaires o : 

Southern Hemisphere February 1980 January 1981 

Northern Hemisphere Septernber 1980. July 1981 

Sampling 

: __• '_ ■ ■ ' • . 

Discussion of samplmg_specifica- . 

tions and consultation January 1977 May 1978 



127 



ii4 ' 



Interhational Sampling Committee 
prepare draft paper for VtAp 

Final sampling design settled 

Sampling manual prepared 

Manual approved iMC 

National Centers draw samples arid 
consult international Sampling 
Committee 

Ihternatlonal SampUng Committee 
report to IWC 

Natonal Centers contact schools 

arid replace refusals 

Southern Hemisphere 
Northern Hemisphere 

Data collection 

Southern Hemisphere 
Northern Hemisphere 

Data Col lectibri Modes 

Consultations with NationaJ 
Centers on potential methods 
of data collectidn 

Report to IMC by Dr. J. Sehwille 

IMC suggestions to National 
Ceriters and return of National 
Ceriter cdmment 

Methods of data collection 
settled 

Priritirig of answer forms (if 
necessary) and dispatch to 
countries 
Pretest 
Post test 

Cdmpjetion of manuals 

Pretest^ : : • " - r-^ r::^:— 

Posttest 



Start 

May 1978 
August 1978 
September 1978 



February 1979 



Qctober 1979 
June 1980 



February 1980 
September 1980 



JariUary 1977, 
July 1978 



September-~1978- 



January 1979 



£x:)rnplete 

August 1978 
September 1978 
February 1979 
February 1979 

October 1979 
October 1979 



December 1979 
July 1980 



January 1981 
July 1981 



July 1978 
AUgUst 1978 

-Decern be r_ 1 9 7 8 



December 1978 



March 1979 
August 1979 
JUrie 1980 



October 1979 
October^ 979^ 
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Start Cbmpletiori 

Data Pr6cessirig_abcL Analysis 
Preliminary Plahhihg: 

Odtlihe of instruments with _ 

approximate number of items December 1978 February 1979 

Outline of cbdebbbks (dummy) July 1978 February 1979 

Preliminary cbnkideratibn of file 

building July 1978 February 1979 

Detailed Plarihing: 

Settle coding of final instruments February 1979 October 1979 

Standardize punching and coding 

fdrrr.s for Dry Run Febru^y 1979 March 1979 

Settle analyses required by iMC October 1978 February 1979 

Settle file building and weighting 

procedures r February ^1979 July 1979 

Update of Codebooks ^ February 1979 July 1979 

International trial of cduhtnes^ 
capacity to produce files and 

undertake standard analyses _ 

(run as part of the Dry RuhJ February 1979 July 1979 

Write programs for basic item 
analyses, univariates, cbrrela- 

tiohsT^-sehool-rjports^ and for 

special multivariate analyses February 1979 March 1980 

Analysed for IMC arid cbUhtries 

requiring assistance April 1980 December 1981 

CbhstrUctibh bf data bank December 1981 becember 1982 

.0 ' [ 
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APPENDIX A 
uflDN OF COCNiTiV 
fiQPULATION A 
176 ITEMS 



CONTENT 




BEHAVIOUR hEVEtS 




TOTAL 




I 


II 


III 


IV 

1 V 




008 ARITHMETIC 


in Items 


19 Items 


15 Items 


2 Items 


60 Items 


001 Whole Numbers 


A/1 

A/ 2 , 
A/3* 


A/4* 
A/5* 


A/6* 
A/7* 


A/8 


8 


002 Common 
Fractions 


Core/I* 
eore/2 

A/g 

A/ 10 
A/ 11 
D/2 


Cbre/3 
Cbre/4 

A/12* 
- A/13 

D/1 

D/3 


A/14 
b/5 
b/4 
b/6 




16 


003 Decimal 
Fractions 


Cbre/5* 
B/1 
B/2 
B/3* 
C/6 


Core/6 
B/4 
C/9 

c/ib 
e/11 


eore/7 
ebre/8 
3/5/ 
e/12* 




■ n 

1? 



C/8 



664 Ratib, Core/9 Core/ll gore/ 4 

Propbrtioh, Core/1 0 Core/12 Core/15 

Percent C/l Core/13 Core/16 i3 

C/2 C/3 C/4 

' ^ C/5 



005- Number Theory B/7 . B/6* B/9 

B/8 



006 'fers C/IS* 

e/14 
C/15 
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CONTENT * BEHAVIQUR bEVELS TOTAL 



I II III IV 

— - • • _ _ 

100 ALGEBRA 2^ Items 15 Items 8 items 1 ^^^^ 48 Items 



1dl Integers D/7 D/11 D712 

D/8 
b/9 
b/10 



102 Ratlohals D/13 SI 



103 Integer ^ D/l4 
Exponents 



lO'l Formulas 



Core/l? 


Gore/19 


eore/2d* 


A/15 


A/19 


A/2e 


A/16 


A/21 


A/22 


A/17* 




D/23 


D/15* 


D/20 


D/m 


D/17 


D/21 


D/25* 


D/18 


D/22* 
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105 Poiynomials 
■ Expressions 



. A/18 
b/16 
Core/ 18* 



D/ig 



106 fej ujt[qris _and^ 
'." Tnequationi 



■Cone/ 21 ^ 
Core/ 2 2* 

B/10 

8/ 11 

B/12 

B/13* 

C/16 



^JJM— — ^Cdre/-23^CbPp/-24- 



B/15 
C/17 
C/18 

e/ig* 



IS 



107 Relations 



e/21 



e/2d 



120 



eSNTENT 




BEHAVIOUR LEVELS 


- 


TOTAL - 


- 


I 


• ii 


ill 


IV 




260 GEOMETRY 


8 Items 






2 Items 


35 Items 


201 Classification 


y 


A/25 







1 


202 Properties 


A/23 


v^ore/ z 7 




Core/32* 




D/27 


D/26 


A/ 27* 


A/31* 






o 




6/28 




10 














203 Congruence 


Core/ 2 5 




Core/ 29 














3 


, 20n Similarity . 




Gbre/28 


Co re/ 31 




3 






A/28* 






205 Geomf^trlc ebristructibris 




S2 




1 

1 


206 Pythagbrean 


Cbre/25 




. A/30 




3 


D/31 








207 Cbbrdihates 


Atm 


S3 


A/29 










S4 









208 - D edtK:t-tons 



209 Trarisfbrmatidn 
(informal) 


C/22 


C/23 


Core/se 
C/24 
£/25 : 


5 


.212 Spatial 

Visualization 


c 


D/32 

V 


A/32 


. 2 


215 transforma- 
tlonal Geometry 


S5 


ss 

57 


S8 








13^ 


I * ; ' 





ERIC 



ERIC 
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CONTENT BEHAVIOUR, LEVELS - TOTAL 



II III IV 



300 PROBABILITY 2 Items 2 Items U Items 1 Item 9 Items 
AND 0 . 

STATISTICS 

. I ■ 

302 drganlzatfori B/22 1 



303 Representation B/19 BflU* 

B/20 n 



B/21 



30i» Mean, Median, B/16* B/23 B/17^ 
Mode 



306 Probability B/18 



400 MEASUREMENT 7 Items 7 Items. 8 Items 1 Item 23 Items 



401 Units C/26 C/29 Core/33 



402 Estimation B/25 = B/28 Core/35 

C/30 



403 Approximation Core/34* 

B/29 



404 Determining eore/36 eore/39 eore/38 C/32 14 

Measures eore/37 B/26 Core/4b 

;B/27 B/30 

C/27 B/31 

C/28 B/32* 



e/3i 



s 



135 



*Anchor: Items 

Item refererices are given by Farm aridJiUmber 
e.g.^ A/3 is item number 3 iri Rotated Form 
"S" items are supplementary ^ added to reflect I, topics 
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DISTRIBUTION OF COGNITIVE ITEMS 

POPULATION B 
136 Items (120 items plus . 1 5 sujaplemehtary items) 



CONTENT 


BEHAVIOUR LEVELS 


TOTAL 




1 li III IV 






: ■ 





1 SETSi RELATIONS 2 Items 3 Items 1 Item 1 Item 7 Items 
AND FUNCTIONS 



1 .1 Set Notation 




1 /I 






1.2 Set Operations 




^ 7 1 * 
1 






1.4 Functions 


4/1 


6/1 


7/1 


07 1 
Z/ 1 


• 


5/1 


■ 






2 NUMBER SYSTcMS 


4 Items 


4 Items 

■ - 


C 1 ^ Ann ^ 

3 iiems 


"3 I'I'ortic 1 fv-J li'oiTic 
^ 1 Lclllb 1 vT 1 Lclll3 


Z.I oommon t^aws 


1 / z 




1 7*^ 




2.2 Natural 










Numbers 










2.3 Decimals 


3/2 




4/3 


7/4 


2.4 Real 




4/2 


6/2* 




Numbers * 








3/4 


2.5 Complex 


5/2 


7/3 


2/3 


Numbers 


7/2* 








3: ALGEBRA 


9 Items 


6 items 


7 Items 


3 Items 25 Items 


3.1 Polynomials 


6/4 


1/5 




1/7 


7/5* 


2/4 










5/6 






3.2 Quotients of 


3/6 








Polynomials 


5/5 








3.3 Roots and 


4/4 


. 7/6 


5/7 




Rad[cals 


6/5 




6/5* 




3.4 Equations and 


3/5* 


.4/5 


7/7* 


2/6 


inequations 


6/7* 




1/6* 




3.5 Systems of 


2/2 


3/7 


3/8 


4/6 


Equations and 






1/4 




Ihequatlohs 










3i6 Matrices 






■■ 2/5 


v 



CONTENT 





i 

. 1 


j j 
II 


j i i 
1 1 1 


1 V 




4 GEOMETRY 


5 items 


4 items 


5 items 


4 Items- 


—1*>-| terns- 


4.1 Euclidean 


7/9 


\ 








*♦ . o Mnaiyi ic 


o/ o 


1/8^ 


4/8* 


?/? 


- 








5/9 


1/10 




•♦•3 vector iViei.nQQS 


H7 / 


7/8 


3/10 


/ / 1 0 








2/7 








H»o 1 rigonuinexry 




6/9 


. 2/8* 


0/ 1 g 










1/9 






t supplementary items J 




SI - SI 4 


- - 


(m Items] 


AM A 1 YQIQ 


ID Items 


13 Items 


12 Items 


ft 1 ^j^rvi^ 

*♦ Items 


ft ft 1 ^ ^ _ 


D«i ciemeriLary 


1 7 1 1 * 


6/12 . 


4/11* 






Fuhctibhs 




5/11 


3/13* 








•3/11 

o/ 1 1 


4/9* 


2/11* 










4/10 


2/12 






D rroperties or 




3/12 


1/12 


7/12 




Functions 




5/10* 






• 


D»p .LimiLb a no 


0/ 1 1 


7/11* 


8/5 






continuity 




8/3* 








5.4 Differentiation 


8/1 ^ 


5/12* 


8/13 








1 / 1 O 










5.5 Appiications of 


2/13 




8/12 


8/6 






H/ 1 ^ 






0/ 1 H 






6/13 












7/14 












8/7 










5.6 Integration 


8/2 


8/9 


7/13 








8/8 


8/11 


8/4* 






5.7 Techniques of 


1/14* 




5/13* 






ihtegratiori 












5*8 Applications of 


2/14 


4/13 


3/14 


8/15 




Integration 




8/10* 








5 PROBABILITY AND 


4 Items 


2 Items 


2 Items 




' 8 Items 



STATISTICS 



6.1; Probability 



2/15* •• 6/14 
3/15* 



124 













CONTENT 




BEHAVIOUR LEVELS 


TOTAt 




i 


jj 


ill IV 




6 i 2- Statistics 
' 6.3 Distributions 


5/m 


1/15 


.4/15 
5/15 




7 FINITE MATHEMATICS 




1 item 


1 Item 


4 Items 


7.1 CombinatDrics 


S15 


6/15 
S16 


7/15;. 





ERIC 
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Appendix B 

STRUCTURE OF POPULATjON A COCNltiVE TEST 
. • [ {LdhgitUdihal) ' 

_ CORE TEST 

HO Items 



Content Behavior Levels Total 



II III IV 















Common 'and 


B/6* 


k/8 


L/22 




8 


- Decimal 


H/5 


G/5 


L/2it 






Fractions 


A/3b 


A/6 








Ratibi ProfDor- 


t/3 


t/23 


H/9 




8 


tion and Percent 


• L/6 


1/8 


E/8 










F/1 


F/25 






. Formulas, 


c/m 


6/12 


J/i6* 


b/28 


8 




D/25* 




L/l4 






ExpressibriSi 


A/17 










Equations 


F/17* 










Geometry 


L/27 


J/27 


HJ2U 


1/26* 


8 




K/20 


K/28 


B/32 












C/27 






Measurement 






C/36 










1/39* 


E/39 








K/30 








8 




K/35 


K/36 


K/27 . 












J/40 






Totals 


13 


11 


^^ 


2 


i*0 



*anchor item ■ 

Items identified by numbers Used in rribst recent set of trial Iristra- 
ments (1978) 



This Appendix has not been updated to show the sUpfjlerriehtary items. 




POPULATION A 


COGNITIVE 


INSTRUMENTS 




... 


- 


176 Items 




- 




. . ._• . 

Content 

. , , ........ 


- — 


test Forms 










Core 


Form A 


■ - 
Form B 


rorm C 


— r- 

Form D 




N items 


N 1 temi 


N items 


N items 


N i terns 


Whole Numbers 




8 




• • 




eommoh Fractions 


8 


6 






6 


Decimal Fractions 




-, ' 


5^ 


7 




Ratio^ ProfD; , Percent 


8 






5 




Number Theory 












Power/ Exponents 








3 • 




Sqtia re Roots 








c 




Dim. Analysis 












lntegers> Rat., Exp. 










8 


Fb rrriulas 




Q 
O 






1 1 


Expressions 


8 






6 




Equations 






6 




Plane Figures ^ 


8 


10 






7 


Trans fdrmation 








fi 

4 




ProbabilltY/Stat. 






9 






Measurement 


8 


8 


7 






Supplementary Items 


8 


2 


2 


2 


2 


Totals 


40 


34 


34 


34 


34' 



ERIC 
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ROTATED FORM A 
32 items 



Content 




Behavior Levels 




Total 






r 


II 


III 


IV 




Whole Numbers 


E/2 
D/1 

U 1 1 

1/2* 


F/2* 

J/ 2 


C/4* 
1 /I * 


e/10 


8 


Common Fractions 


Din 
\m 

J/3 


K/13 


K/31 




6 


Formuias , Expressions 


K/32- 
H/16 
B/15 
1/16 


J/15 
* t/16 


C/17 
B/19 




8 


Geometry 


1/31 
6/28 


1/23 


B/28* 
1/24* 
5/23* 
G/24 

D/34.: 

A/31 


F/33* 


.10 


Totals 


12 


7 


11 


2 


32 



*anc r item 

Items identified by numbers used in most recent set of trial 
instramehts (1978). 
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ROTATED i=dRM B 
32 items 











Total 




'■ Gbnterit 




Behavior Levels . 








1 


II 


III IV 








B/7 
Qll 
C/7* 


1 /7 • 

lit 


C/6 


5 




Number Thepry 


A/9 


BJ2* 
1/9 


D/2 


4 




Algebra Equations/ 
Ihequ. . ; ' 

Reiatibhs 


,B/i3 
n / 1 o 
0/19 
E/21* 


E/22 
r / 1 y 




6 
















Prbbability/Stat. 


H/33* 
H/35 


A/36 
C/33 


F/36* B/34* 

F/39 

6/34 

G/35 


9 




Measurement 


E/37 
t/9 


A/39 
B/31 
B/39 


J/39 
1/37 
B/4e* 


8 




Totals 


12 


19 


8 2 


32 





*anchor item ^ - , - ^ 

Items identified by numbers used in most recent set of trial ihstrumehts 
(1 978) ; 
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ROTATED FORM C 
32 items 





Gbhteht 




Behavior L. 


iveis 




Total 

r 




\ II III IV 


Ratib^ PrbfDortioh^ 
Percent 


K/ib 

D/13 


B/8 - 

3 


G/9 
E/9 


. - 


5 


Decimal Fractions 


K/l, 

F/6 

D/9 


C/40 

K/2 

F/7* 


J/11* 




7 


Powers /Exponents 


C715* 
L710 
E/l 1 




A 


3 


Algebra EqUatidris 

Inequ. 
Relations 


K/7 
1/28 


i/17 
C/19 
K/26 
Added* 






6 


Trans. Geo. 


J/ 32 


C/25 

ft 


E/35 
B/24 




n 


Measurement 


F; 37 
K/39 
b/26 


K/11 
F/38 


L/7 


K/40 


7 


Totals 


14 


11 


6 


1 


32 



*anchorjtem 

Items jdent if ied by numbers used in most recenit set of trial 
Instruments (1978) 



13D 



ROTATED FORM D /4^> 
3i items 



Cdhterit Behavior Levels Total 





1 


II 


III IV 




Common Fractions 


A/3 


A/J» 
L/1 


D/7 
L/8 
B/5 




Integers /Rational 
Integers 

Rati6nals_ 
Integer Exp. 


Hf^^ 

1/12 
E/13 
A/11 
H/15 

KfH . 


D/18 


H/13 


8 


Algebha Formulas 

\ 

N 

Expfessjons 
Equations 


' Added* 
L/15 
G/15 
D/22 


Din 

-A/14* 
F/15 


E/18 
L/33 
Added* 


11 


Geometry 


d/31 
b/36 : 


A/21 

K/ig 


F/24 
G/30 
H/UO 


7 


Totals 


13 


Q 


10 0 


32 


*anchor item 
Items identified by numbers used in 
(197a) 


most recent* set of trial 


■ instruments 



in 



STRUCTURE OF THE COGNITIVE TESTS— POP; B 

f LbngltudihaJ) / 
MARCH 1979 / 



TEST FORM 1 TEST FORM 2 





1 


II 


. Mi 


IV 


i 


/ 

w y 


1 1 1 IV 


Sets and 
Functions 




1/3 










14/8 


Number 
Systems 


6/1 




1/4 




Mil 




10/10 


Algebra 


10/1 


8/8 


5/7@ 


14/13 




1073 


2/6 3/4 


Gebrhetry 




2/7(1 


11/11 


18/14 




14/5 


4/1 5@ 14/4 


















Analysis 


8/1 4@ 


2/14 






2/10 




12/12 
'12/l4@ 


Calculus 


11/14. 
.1/14@ 








5/19 
10/18 






Prdbability , 
Statistics & 
Finite Math. 


11 /20 






1/1(S' 






item Count 


. 5 , 


4 


4 . 


2 


5 




2 5 . 3 



Items marked- @ are anchor items from the First Sttjdy. 



H5 



II : II! IV 



II III IV 



Sets and 2/2@ 3/1 

Functions 



N urnbe r 
Systems 


6/2 . 




7/3 10/9 






6/3 


2/4 


AlgeBra 


New @ 
14/3 


9/7 


8/9 




6/6 


1/7 


10/4 


Geometry 


11/19 




12/9 




4/1 3@ 




ll/12@ 


Analysis 


10/7 


8/1 


4/160 




5/56 
2/13 




7/1 5@ 


Calculus 






9/20 


1/11 


2/15 






















Probability^ 
Statistics S 
Finite Math. 


5/9@ 






14/1 




2/1 




Item Count 


6 


3 


5 1 " 




5 


5 


4 1 



Items jriarked Q are anchor items from the Rrst Study. 
Test Form 3 includes one anchor item recently added to the pool, with- 
out prior piloting in the Second Study. (Form C, Item 18, Husen p. 329) 



'I - 





Test Form 5 Test Form 6 




i II 


111 IV 1 


II 


III IV 


Seii and 


13/1 




2/3 




Functions 









Number 


8/a 9/3@ 


6/i»@ 


8/2 


8/3@ 


Systems 











Algebra 


11/7 Q/3.. 


3/11 11/5 




3/169 






14/7 










1/9 







VJcUlllc; LI y 


8/ 1 


5/10 , 12/7 


3/1 3 


6/13 


Analysis 


2/5@ 


5/16@ 


10/5 




13/19 








Calculus 


5/17 


6/150 12/16 






Probability, 


12/11 


13/20 


3/19 




Statistics S 




0 


3/20 




Firilt# Math. 








item Count 


5 5 


4 1 6 


6 


2 1 


Items marked 


@ are anchor 


item's from the First Study. 







ERIC 
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Test Form 


7 




test 


Form '8 


(Calculus) 




j 


ii 


Mi 


IV 


i 


ii 


iii IV 


Sets arid 
Functions 






10/8 , 










Number 


7/15@ 


13/2 




14/2 








Systems 
















Algebra- 


8/5@ 


9/4 • 


4/5@ 










Geometry 


^ — 

6/10 


6/12 




13/9 








Analysis 


1 5/9 


11/13 




13/11 


2/12 
3/17 


4/1 9@' . '5/110 7/18 
15/16 


Calculus 


1/10 




12/18 




14/19 
15/5 


15/4 

2/7 

14/17 


14/15 15/13 
.15/3 15/2 


Probability^ 
Statistics S 
Finite Math. 






6/20 




< 






Iteni Count 


5 


4 


4 


3 


4 c 


4 


4 3* . 



Iteifris marked @^ are anchor items from the First Study 




O ■ - • . : 

ERIC 
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App endix X 
ANCHOR ITEMS 
POPUt-ATidN A 



Final Most - 





Draft 


Recent 


First 




1 Ur 1 L 


Item No. 

■ 


Trial No. 


Stady 


Changes 


Whole Numbers 


A/3 


1/2 


A/ 3 


Nil 




A/S . 


F/2 


A/17 


Nil. 




A/5 


J/2 


B/5 


Nil 




A/B 


CJH 


B/2 


Nil 




A/7 


L/1 


A/2 


Nil 




.B/6 


B/2 


A/23 


Nil 


Fractions 


CPCI/1 


B/6 


B71 


Nil 




A/1.2t 




A/^ 


Nil 


Decimals 


CPCI/5 


A no 


C/2 


Diagram 










above ^ 




B/3 


C/7 


B/3 


bivisibri 










above 




C/11 


F/7 


^ B77 ' 


Nil 




C/12 


j7ii 


C/7 


Nil 



Powers e/13 C/IS A/6 Nil 







c 






Formulas and 


CPei/18 


D/25 


B/11 


Nil 


Expressions 


C PC 1/28 


J/16 


B/23 


"cents" 










added to 










responses 




A/17 


B/15 


B/14 


Nil 




B/15 




B/15 


Nil 




D722 


A/14 


B/.9 


"and c" 
added to 










stem 




0725 




A/11 


Units 



Equations arid CPCI/_22 F/17 A/m Nil 

Iriequatibris B/13 E/21 A/12 "is equiva- 

_ _ • lent to" 

C/19 - B/10 Nil 



TOPIC 


hinal 
Draft 
Item No. 


Most 
^ Recent 
Trial No. 


• First 
Study 


Changes 


Geometry 


GPCI/32 
._A/26 

1^1.4,1 

A/28 
A/31 


1/26 
B/28 

Earlier Trial 
F/33 


A/lb 

e/2b 

B/13 

e/6 
e/22 


Diagram 
above 
Line symbols 
Nil 

Units 
Diagram 
above 


Statistics . 


B/15 
B/17 
8/21 


H/33 " 

F/36 

B/34 


B/4 
B/8 
A/5 


Nil 
Nil 

Graph above 


MeasUremeht 


CPCI/34 
B/3^ 


1/39 
B/4d 


C/8 
A/8 


Scale above 
Units 



Total 32 items 
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Append be 



1^NCHUR~ITEHS" 



POPULATION B 



EKLC 



TOPIC 



Final 
Draft 
item No, 



Most 

Recent First 
Trial No. Study 



Changes 



1 Sets^ Rejations 
and Fanctjqns 
1.2_Set Operations 

2 Namber^^ystems 

2.1 Common taws 

2.2 Natural 5/4 
Numbers 

2.4 Real Numbers 

2.5 Complex 7/2 
_ _ Numbers 

3 Algebra 

3.1 Polynomials 

3.3 Roots arid 
Radicals 

3.Q Equations arid 
IheqUatibris 
3/5 - 

4 Ceombtry 

n.3 Analytic 1/8 
Geometry Q/8 

4.5 Vector Method 

4.6 Trigonometry 

2/8 it/15 
> Analysis 

5.1 Elejnentary ^ 

FurTcf ion's 



3/1 



2/2 



C/19 X,Y,X P,Q,R 



5/3 


9/3 

6/n 


6/2 


6/7 

Nil 


Nil 


6/2 


8/3 

7/5 


5/15 


6/4 

Nil 


Nil 


7/5 
6/6 


8/5 
'3/10 




7/2 
8/2 


. Nlj 
C "and" 


1/6 
7J7 
C/18 


5/7 
4/5 
Nil 




6/11 
6/16 


Nil 
Nil 


?/7 


2/7 
11/12 

4/13 


9/1 
9/7 


Nil 
Nil 
9/13 


Nbtatibri 



"or' 



5/8 



1/11 

W9~" 



2/11 
3/13 
4/11 



12/14 

4/16 

7/15 



8/13 5/9 

8/4 Nil • 

^B/14 6/9 

5T5~ 5Tn~~ 
5/16 Nil 
5/19 Graph above 
6/lG Nil 



re above 



Nil 



■"^Rahge" deleted 



5.2 Properties of 


5/10 


2/5 


5/6 


Nil ' 


Functions 










5.3 Limits and 


6/11 


5/16 


6/15 


. Nii 


Contirialty 


8/3 


4/19 


9/15 


Nil 


5.4 Differehtiatioh ' 


5/12 


5/17 


9/3 


Nil 


5iB iritegratioh 


8/4 


5/11 


9/9 


Notation ' 


5.7 Techniques of 


1/14 


1/14 


9/5 


Nil 


ihtegratibh 


5/13 


6/15 


9/4 


Nil 


5.8 Appiieatiohs of 


8/10 


2/17 




9/10 Graph above 



Ihtegratibh 
PrbbabiJity and Statistics 
6.1 Probability 2/15 



1/1 



8/3 



Nil 



TOTAL 30 Items 



APPENDIX S 

Final Fbim, Population A Test ( Cross-Sect idnalj 



CORE FORM 





t -- . 






■ 


■ 


■ 

Code ' 






Diff, 




NcK — - 


— Cont- 


• Beh. 


Diff . 


Item 


No. 


Cbht . 




^ ■• 1 


OlU 


102 


ii 


53 


. 21 ■ 


033 


303 


II 


60 


2 


02t 


206 


I 


31 


22 


018 


106 


IV 


UU 


3 


.006 


003 


- Ill 


5^ 


23 


010 


oo5 


1 


57 


k 


' 012 


101 


I 


65 


lk 


037 


UoU 


111 


37 


5 


030 


•209 

_ _ . 


I . 


kk 


25 


016 


lOU 


111 


53 


6 


025 • 


203 


Ill 


39 


26 


008 


odU 

30 U 


II 


48 


7 


032 


302 


II • 


50 


27 


035 


I_ 


Zk 

tr ^ 


8 - 


OiiO 


koh 


III 


3U 


28 


021 


201 


II_ 


55 


9 


^ 022 


202 


I 


. 70' 


29 


029 


207 


III 


4l 


• 10 


015 

• - 


loU 


I 

• 


57 


36 


036 


Uoi 


I 


50 


. 11 


. 039 


kok 


i 


6k 


31 


002 


001 


IV 


58 


12 


- 017 


106. 


I 


63 


• 32 


023 


ono 
d\)d 


1 V 




13 


031 

ooU 


212 


TT T 
111 




33 


009 


00^ 


I 


55 


Ik 


002 


11 


76 


3U 


Oil 


008 


II 


30 ■ 


15 


03U 


303 


111 


66 


35 


007 


003 


Ii 


6U 


±6 


. 020 


110 


. Ill 


2*2" 


■ 36 


02U 


:203 


III 


52 


17 


003 


002 


I 


68 


37 


038 


U02 


II 


US 


18 


005 


003 


I 


52 


38 


028 


207 


I 


56 


19 


026 


20U 


III . 


69* 


39 


019 


107 • 


11 


6U 


20" 


001 


001 


1 


85 

- - 


llO 


013 


, 101 


111 


51 






















'-MHacrr 










(N = iiO) 






5U.25- 


Diff.- 
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3xnt,--Po-p ulation A Test ( Crbss-Sectibhal ) - Cbritiriued 



FORM A 



Item 



1 
2 
3 
If 
5 

6 
7 
8 
9 
10 

11 

12' 
13 
il+ 

15: 

16 
17 
18 

19 
20 

21 

22 
23 

2U_ 



25 

26 
27 
28 
29 
30 

31 
32 
33 
3k 



Mean 
Diff. 



Code 
No. 



eont. 



Beh. 



052 
055 
050 
062 
072 

173 ■ 
067 

057 

■ OUU 

0k3 
069 
058 
06h 
031 

0h6 
QhB 
063 
055 



071 
066 



070 



ohi 
056 
053 
061 

068 
05^ 

0i*2 
059 



107 
101 

208 

. U03 

215 

30U 
207 
201 
002 

002 ^ 

U02 

202 

211 

102 



OOU 
006 
209 
301 



003 
101 



koh 

20 k 
001 
110. 

105^'- 
207 

1+01 

.106 

001 
202 



Hi 
III 
III 
..III 
lii 

I 

III 

li 

III 
Hi 

II 
II 
II 

I ' 
IV 

I 
I 
I 

II 
II 



II 
I 

li 

I 



I , 
I 
I 
I 

II 

I_ 
II' 
I 

IV 



Diff, 



U3 
52 
83 
39 
i47 



55 

71 
69 

52 
63 
36 
8U 
^5, 

62 

n 

-30 
58 
70 



ho 
70 

h9 



57 
88 

58; 

5U. 
59 

.80 
58 
79 
'33 



Item 



1 
2 
3 
k 
5 

6 
7 
8 

9 
10 

11 
12 
13 
lU 
15 

16 

17 
18 

19. 
20 

21 
22 
23 



25 

26 
27 
28 

29 
30 

31 
32 
33 
3h 



Code 
No. 



Cbnt . 



086 

100 

093- 

085 

088 

10 U 
Ofh 
073 
176 

080 
078 . 
083. 



090 

089. 
101 
097 
099 
171 

103 
081 
077 

091 



075 
076 
087 
Q8h 
102 



092 
082 



Beh. 



Diff, 



106 


I 


U5 


■ Ugi 


I 


- 5^ 


206 


I ■ 


30 " 


lOii 


II ■ 


63 ■ 


105 


I 


28 


208 


ii 


: 51 : 


kok 


III 


62 


001 


iv 


36 


001 


I 


87 


205 


III 




.005 


I 


60 


003 


III 


1»3 


102 


ii 


53 


20 U 


li 


38 


202 


I 


'■■ sit 


201 


ii 


71 




Hi 


72 


302 


I 


3k ■ 


30 U 


IV 


k9 


215 


• Ii 




Uok 


III : 


65 


008 


II 




003 


II 


61 




^~'^il— - 


—ke— 


203 


I . . 


81 


002 


: li 


65 


002 


III 


k6 


I06 


III 


35 


■ 103 


I 


28 


U02 


li 


k9 


303 


II 


59 


2Qk 
207" 


III 


67 


III 


50 


±0i . 


II 


•in 



(N = 32) 59. kk 



'W:= 32) 



52. Q3 
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Final Form, Poptaation A Test ( Cross-Sec tibhal4- - Continued 

■ FOR^l e . 1™ 2. 



— It (5m 


Cod^ 
No. 


Cont . 


Beh. 


^f-f^ 


-item 


Code 
No. 


Cbht. 


Belt. — 


M.ff ^ 


1 
2 
3 

5 


llU 
130 
175 

133 


' 101 
30U 
207 

UU J 

Uoi 


IV 
i 

11 

i 

HI ■ 


^^5 
k3 

1+5 

. 59 


1 
2 
3 
k 
5 


159 
. ■ 155 ■ 
ll+l 
151+ 
1I+6 


212 
202 
603 
201 
009 


11 
III 
III 
III 

I _ 


^_52 
58 

67 . 

U2 

77 


6 

T. 
8 

9 

. 10 


136 
125 
• 132 

129- 


UoU 

20li 
30 U 

UU-L 

303 


II 
II 

1 

II 

111, 


33 
30 
50 
38 
3iJ 


6 

7 
8 
9 
10 


ll+O 
: II+3 
I6I+ 
■ 153 
150 


003 

doi+ 

1+02 
110 
105 


I 

III 
I 

IV 
1 


5i*- 
79 
63 
55 
59 


11' 

12 
13 


135 
109 
126 
131- 
116 


U03 
003 
207 

loU 


II_ 

III 

11 

T T 

I 


1+8 

74 
ilO 

kk 


11 
12 
13 

. 1:1+ 
15. 


163 

: II+7 
II+5 
138 
139 


306 
102 

. 008 
001 

. 002 


I 
■:l 
II 
III 
I 


U2' 
U8 
60 
7U 
61 


■ 16 • 

IT 

18 
■ 19 
20 


107 
119 
121 

n oft 
l^Q 

110 


002 
lOJ 

2gi 
ooU 


I 
I 
I 

III 


63 
6o 
66 

58 


16 
17 
18 
19 
?0 


1I+I+ 
168 
166 
11+8 

16'J' 


005 
1+0 1+ 
1+01+ 
• 161+ 
1+02 


IV 
IV 

I . 
. 11 
ill 


5U 
21 
^29 
' 60 
51 


21 
22 

^23 


120 
112 
113 


lio 
oo6 

J.UJ. 


11 
1 

J. 


\3 
6o 
52 


21 
22 
23 
21+ 

25 


152 
137 
162 
171+ 
157 


107 
• 001 
■ 30l+ 
215 
208 


I 

II 
II 
11 
II 


75 
63 
58 ■■■■ 


- 2U 

25 


172 
127 


" 103 
. 208 


II 

III ; 


,1+5 


55 ^ 


27 
28 
29 
.30 


1 oil 

123 
117 
: 13U 
• 122 


202 

ie5 

U02 
202 


• II 
III 

II-- 

iii 

II.. 


■59 
31+ 
5^ 
36 

79 


26 

27 

28 

29 
30 


11+2 
17Q 
165 
155 
151 


ooU 
.215 
UgS 
20 U 
106 


II 

■ 

II 

■ III- 
I 


- 29 

— D 

^5 
55 
1+5 


31 

. " 32 
33 
3k 


■ 105 
115 
111 
■ 118 


001 

loU 

005 
lo6 


: III 

I . 

II . 

I . 


71 
28* 

78 
50 


' 31. 
32 
33 
31+ 


.11+9 ■ 
158: 
160 
i6r 


igU 
209 
303 
303 


*iii 
111 
II 

■ 11 


72* 

60 , 

57* 
1+8* 


■ Mean 
• Diff. 




(N = 32) 




1+9; 09 






(N ^ 


32) 


55^69 
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mth-Pdpizlatibh A Test ( Cross-Sect ionai) - dcntinued 



CORE FORM 





.... 1 
Level j 


Content 


. • • I ■ 


■ II 


III 


IV i 


boo 


" 5 


5 


1 


-4 

1 i 


^ 100 


3 


1 


3 


i 


200 . . . 1 


h , 


1 


6' 


1 ■ 


300 : 


: 1 


2 




Q 


uoo ; 


2 


1 


2 


Q 


Total 

1 ^ : 


15, 


10 


12 


3 ; 

*• ■ » 







FORM A FORM B 





tevel 






Level 




Content 


I 


II 


III 


IV . . 


I 


II 


III 


IV 


000 


k 


3 


1 


0 


2 


3 


3 


1 


100 


3 


X 


3 


1 ■ 


3 


3 


1 . 


0 


200 


k 


3 


2 


1 


3 


h 


3 


b 


300 


0 


1 


■ : .2 . . 


0 


■ 1 


1 . 


0 


1 


UO0 

f 


2 


2 


1 


0 ' 


1 


1 


3 


0 


: Total 


13 


10 




2 


10 


12 . 


10 


2 










FORM 


D 






Level 






Level 




Corit ::nt 


I 


II 


III 


IV ■ 


I 


II - 


III 


IV 


















000 


3 


2 


3 


6 


3 


3 


3 


1 


100 ■ 


5 


3 


0 


■ 1 


i| 


1 


1 


1 


200 


1 


3 


3 . 


0 


P 


3 


5 


0 


: 300 . . ; 




1 


^ 1 


0 


1 




0 


0 


Uoo . i 

. 1 — 


0 


'2 


2 


0 


2 


1 . 


1 


1 


Total : 


11 


13 


9 


1- 


10 


■'ii 


10 


3 












if5 









ERIC 



139; 



APPENDIX — R 



FINAL FORM 
POPULATION B TEST (CROSS-SECTIONAL) 



o 

ERIC 



. Population B - Final form - Cross-sectional 

FQll • : : ---_EQBii- 



Code, 

lo. 'Cont. Beh. Diff. 



02T 
096 

099 
103 
015 



5.riii 
3.3 n, 

if.l I 
5:6 III 
5.T II 



■56 
5? 



129 
028 

059 
093 

,0JO 



7.1 ill 

6.2 I' . 
2.5 ii 
5.2 ii 



128 

0T9 
032 

005 
0T6 

100 
081t 



1.2 III 
3.1 I 

2.3 I 

3.5 III 
l.li II 

I? 

1.6 II 



61 
82 
39 
59' 



T2 
:52 
?3 
59 

35 
T2 



Code 

So. Cont. Beh. Diff. 



■FOIiM-3- 



002 
006 
13ii 
012 
055 



2;.l 

■3.1 

1.5 

5.2* 
5.1 



I-^ 
III 

I_ 

n 
II 



086 
116 
05c 
069 
058 

001 
0^5 
121 
066 

roll 



080 



5.3 I 

5.6 II 

i,\ III 

:l;3 III 

5.8. II 

i;2 il : 
3. ' 



i 

IH 



3.1 I 
5. '5 I 



63 
59 

mm. 

63 
lili 

5? 

ill 

\5 
83 

,!i8, 

86 
T9 

91 



■Code; - 
Ne^ Gent-. Beh. Diff. 



2.1 iii ii9 

3.3 I , ^6T 



I :ir III IV. 



I II III IV 



082 
062 
115 

.051 
Oil 

031 
033 

0ii9' 
123 
008 

135 
020 
056 
061 
106 

iii 



3.1 i 

,2.5 : 
5.8 Ii 

5.5 III 
6.2. II 

1.2 II 

2.2 :'HI 

3.3 I ■ 

\, III 

3.1. II 



81* 
33 

35 

31; 

8T 
li9 
66 

35. 



3. II 
3.6 III 
5.1' III 
1.1 r 

"yX I 

5;5: iV 
1*.3 ill 



if5 
li6 
TO 
13 

\l 
?3 



1 , ii iii iv 



Code _ . 

No. Cont. Beh. Diff. 



m 

019 
025 
113 
091 

12| 
023 
131 
01*8 ■ 

05I1 



i.ii I 

3.1' Ii 

1.6 i 

5.? ii 

IX III 

3.. I . 

1.6 III 

I.: IL' 

2.3 ill 

5.1 II 



089 
065 
092 

030 
012 



6.1 III 

3.2 I 

2.5 I 

6.1 I 

5;1 Hi 



119 iV 
lie 5.3 III 



9T' 
11 



n 



29" 

62 

38' 
It 
83 
55 
53 

51' 



I II III iV 



X 9 



1 
1 
1 



1 

1 
.•1 
1 

1 



., 1 
1 , 1 

.:2 



1 

3 
1 



^ — 3- 



1 
1 
2 

. ,1 



6 . 5 ■ 5 1 



1 
1 

2 • . 
2 1 



T ■ 5 • 5 



5 5.61 



2 

1 

r 



1 

2, 
1 



6 li' 6 1 



Mean ■ . 



(15) 61 



(15) 5if 



(13) 62 

158 



EopiiIsti5D4 - Fiiial Forms - Crdss - s e ctloh al i page '2. . 
' FQl S' ; FORM 6 



■ I 
.2 
. 3 
4 



eode ' 
i. Cont. Bell. Diff. 



0T8 2.2,; II 30 

e6l : 2;2 11 .55^ 

e6T 3.3 m. '|3 

109 5.^ ni :52 

G35 ■ 3.1 K 



■ nil 

039 
132 
090 

05T; 



ii 

U I 
I 11 

?.i ii 
3.5 ill 



28 
59 



73 



102 

112 
022 
118 



5.2 IV 

U III 

5.5 . 1 . 

lyii 
•5.1 in 



095 3.1 I 
122 5 II 



^2 

38 
58 

•^0 

< 



Code , 

No. Coiit. Beh, Diff. 



FOl? 



OOl 
101 

m 

Oil 
010 



3.5 I ■ 

5.3 II 

2.i III 

5.1 I 

U III 



13 
65 
15 
62 

III 



130 
02i| 

053 
068 
060 



I I 

if. 3 IV 

1.3 HI 

1.6 I 

•6.2 III 



T3 

IT 
to 

19 



Code. : ; 
M' - -C-9Bt.-MT-Di-ff-.- 



08i:' 

08f 

105 

018 

01*3 



3.3 HI 

1.3 i 

T.i m 

2.5 Hi 

5.1 Hi 



65 
66 

33 

50- 
i2 



09r 

m. 
121 
051 

0T3 
10? 



2.3 IV 

3.t HI 

5.8 m 

I1.3 i 

3.5 iV 

5.T HI 

5.6 I 



•68 
53 
38 

53 

21 
T6- 



058 
108 
126 
OIT 



\l 111 

3.5 HI 

5.3 n 

1.2 n 

3.5 



1*8 

61 
29 

15 



UT 
Olil 

ou 

133 



5.5 IH 
5.1 'I 
lf.3 I 

lt,.2',„I , 

If. 5 ir 



62 
83 

It 

T2 



OlT '2.1* IH 
013 '5.2 'i 



5T 



Code ■ ' 

Jo.— :ecnt,Jeh.. fliff;j:: 



0T8 
0T5 
029 
036 
136 



5.1 n 

6.3 III 
5.8 I 

3.2 'I' 
1.5 H 



8T 
31 

5T 
86 



120 
063 
Gi6 
00? 
125 



5.8 I 

2.1 n 

iVli iV 

;3.1t iV 

7.1 H 



60' 

59 
72 

19 



052 : 15 I 65 

012 . 5.2 II 56 

021 3.I1 IV li9 

i- U IV 28 

026 5.1' Hi 68 



03t 
071 



2.5 IV 

5.1 JI 



19 
53 



I H iH iV 



I II HI IV 



I H III iV 



i n IH IV 



1 1 

1 12 ., 

12 1 
1 2 2.1 



i 

3 

2 



1 
1 
2 

1 2 
1 



' 3 .7 J 2 



6 1 T'"' 3 



1 

3 2 

2 ■ i 



2 
2 

3, 



1 

1 

i ' 1 
2" 3 



1 
1 



1 

i 



6 '3 8 



I 6:2 5 



